Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation

ABSTRACT

A surgical instrument navigation system is provided that visually simulates a virtual volumetric scene of a body cavity of a patient from a point of view of a surgical instrument residing in the cavity of the patient, wherein the surgical instrument, as provided, may be a steerable surgical catheter with a biopsy device and/or a surgical catheter with a side-exiting medical instrument, among others. Additionally, systems, methods and devices are provided for forming a respiratory-gated point cloud of a patient&#39;s respiratory system and for placing a localization element in an organ of a patient.

FIELD OF INVENTION

The present invention generally relates to devices and methodsassociated with a medical procedure, and, in one embodiment, to medicaldevices for use in and methods associated with the respiratory system.

BACKGROUND

Image guided surgery (IGS), also known as image guided intervention(IGI), enhances a physician's ability to locate instruments within apatient's anatomy during a medical procedure. IGS can include2-dimensional (2D), 3-dimensional (3D), and 4-dimensional (4D)applications. The fourth dimension of IGS can include multipleparameters either individually or together such as time, motion,electrical signals, pressure, airflow, blood flow, respiration,heartbeat, and other patient measured parameters.

Existing imaging modalities can capture the movement of dynamic anatomy.Such modalities include electrocardiogram (ECG)-gated orrespiratory-gated magnetic resonance imaging (MRI) devices, ECG-gated orrespiratory-gated computer tomography (CT) devices, standard computedtomography (CT), 3D Fluoroscopic images (Angio-suites), andcinematography (CINE) fluoroscopy and ultrasound. Multiple imagedatasets can be acquired at different times, cycles of patient signals,or physical states of the patient. The dynamic imaging modalities cancapture the movement of anatomy over a periodic cycle of that movementby sampling the anatomy at several instants during its characteristicmovement and then creating a set of image frames or volumes.

Although significant improvements have been made in these fields, a needremains for improved medical devices and procedures for visualizing,accessing and manipulating a targeted anatomical tissue.

SUMMARY OF THE INVENTION

Among the various aspects of the present invention may be noted devicesfor use in and methods associated with medical procedures; such devicesand methods, for example, may include devices and methods that enhance aphysician's ability to locate instruments within anatomy during amedical procedure, such as image guided surgery (IGS) or image guidedintervention (IGI) and such devices and methods may further includedevices and methods that facilitate accessing and manipulating atargeted anatomical tissue.

Briefly, therefore, one aspect of the present invention is a method formodifying or deforming a segmented image dataset for a region of arespiratory system of a patient to the corresponding anatomy of apatient's respiratory system. The method comprises (i) forming arespiratory-gated point cloud of data that demarcates anatomicalfeatures in a region of a patient's respiratory system at one or morediscrete phases within a respiration cycle of a patient, (ii) densityfiltering the respiratory-gated point cloud, (iii) classifying thedensity filtered respiratory-gated point cloud according to anatomicalpoints of reference in a segmented image dataset for the region of thepatient's respiratory system, and (iv) modifying the segmented imagedataset to correspond to the classified anatomical points of referencein the density filtered respiratory-gated point cloud.

Another aspect of the present invention is a method of preparing asegmented image dataset to match the anatomy of a patient's respiratorysystem. The method comprises forming a respiratory-gated point cloud ofdata that demarcates anatomical features in a region of a patient'srespiratory system at one or more discrete phases within a respirationcycle of a patient. The method further comprises density filtering therespiratory-gated point cloud, classifying the density filteredrespiratory-gated point cloud according to anatomical points ofreference in a segmented image dataset for the region of the patient'srespiratory system, registering the classified respiratory-gated pointcloud to the segmented image dataset, comparing the registeredrespiratory-gated point cloud to a segmented image dataset to determinethe weighting of points comprised by the classified respiratory-gatedpoint cloud, distinguishing regions of greater weighting from regions oflesser weighting and modifying the segmented image dataset to correspondto the classified respiratory-gated point cloud.

A further aspect of the present invention is a method for simulating themovement of a patient's respiratory system during respiration. Thesimulation method comprises (i) forming a respiratory-gated point cloudof data that demarcates anatomical features in a region of a patient'srespiratory system at one or more discrete phases within a respirationcycle of a patient, (ii) density filtering the respiratory-gated pointcloud, (iii) classifying the density filtered respiratory-gated pointcloud according to anatomical points of reference in a segmented imagedataset for the region of the patient's respiratory system, (iv)creating a cine loop comprising a plurality of modified segmented imagedatasets through multiple modifications of the segmented image datasetto correspond to a plurality of classified anatomical points ofreference in the respiratory-gated point cloud over the respirationcycle, and (v) displaying the cine loop comprising the plurality ofmodified segmented image datasets over the patient's respiration cycle.

A still further aspect of the present invention is a surgical catheterfor use in medical procedures. The surgical catheter comprises anelongate flexible shaft having a longitudinal axis, a proximal endportion, a distal end portion, and a handle attached to the proximal endportion. The elongate flexible shaft further comprises an outer wallextending from the proximal end portion to the distal end portion. Thesurgical catheter further comprises a biopsy device at the distal endportion, and an actuation wire extending from the proximal end portionto the distal end portion to operate the biopsy device. Additionally, asteering mechanism is connected to the steering actuator wherein thedistal end portion may be moved relative to the proximal end portion bymanipulating the steering actuator.

A still further aspect of the present invention is an apparatuscomprising a steerable catheter comprising a biopsy device for accessingor manipulating tissue.

A yet further aspect of the present invention is a surgical catheter fornavigated surgery, the surgical catheter comprises an elongate flexibleshaft having a longitudinal axis, a proximal end portion, a distal endportion, a side exit in the distal end portion, and a handle attached tothe proximal end portion. The elongate flexible shaft further comprisesan outer wall extending from the proximal end portion to the distal endportion, and an electromagnetic localization element at the distal endportion. A medical instrument housed within the elongate flexible shaftthat is extendable along a path from a position within the outer walland through the side exit to an extended position outside the outerwall, the medical instrument being disposed at an angle of at least 10degrees relative to the longitudinal axis at the side exit when in theextended position. The position of the medical instrument along the pathcan be calibrated to the location of the electromagnetic localizationelement and displayed by a surgical instrument navigation system.

A further aspect of the present invention is a method of guiding asurgical instrument to a region of interest in a patient. The methodcomprises displaying an image of the region of the patient, inserting aflexible lumen into the region of the patient, inserting a surgicalcatheter comprising an electromagnetic localization element into thelumen, navigating the surgical catheter to the region of interest,detecting a location and orientation of the electromagnetic localizationelement, displaying, in real-time, a virtual representation of thesurgical catheter and the medical instrument superimposed on the imagebased upon the location and orientation of the electromagneticlocalization element, and performing a medical procedure at the regionof interest.

A further aspect of the present invention is a method of placing alocalization element in an organ of a patient for use in a medicalprocedure. The method comprises attaching a first localization elementto tissue in a region of the organ of a patient using an endolumenaldevice. The attached localization element may be separate from theendolumenal device and is registered to a segmented image dataset. Thebody of the patient may then be modified such that the body does notmatch the segmented image dataset, and the position of the firstlocalization element is identified from outside the patient's organusing a second localization element to facilitate a medical procedure.

Other objects and features will be in part apparent and in part pointedout hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present invention, both as to its construction andoperation can best be understood with reference to the accompanyingdrawings, in which like numerals refer to like parts, and in which:

FIG. 1 is a schematic illustration of an exemplary surgical instrumentnavigation system according to an embodiment of the present invention;

FIG. 2 is a flowchart that depicts a technique for simulating a virtualvolumetric image of a body cavity from a point of view of a surgicalinstrument positioned within the patient according to an embodiment ofthe present invention;

FIG. 3A is an illustration showing an initially collectedrespiratory-gated point cloud and FIG. 3B is an illustration showing arespiratory-gated point cloud registered to a patient's respiratorysystem according to an embodiment of the present invention;

FIG. 4 is an exemplary display from the surgical instrument navigationsystem according to an embodiment of the present invention;

FIG. 5 is a flowchart that depicts a technique for synchronizing thedisplay of an indicia or graphical representation of the surgicalinstrument with the cardiac or respiration cycle of the patientaccording to an embodiment of the present invention;

FIG. 6 is a flowchart that depicts a technique for generatingfour-dimensional image data that is synchronized with the patientaccording to an embodiment of the present invention;

FIG. 7 is a front view of a patient during the generation of arespiratory-gated point cloud in the respiratory system of a patientusing a catheter and a top panel illustrating that the respiratory-gatedpoint cloud may be taken over the entire respiratory cycle of thepatient according to an embodiment of the present invention;

FIG. 8 is a flow chart depicting the deformation of a segmented imagedataset to a respiratory-gated point cloud according to an embodiment ofthe present invention;

FIG. 9 depicts an exemplary real-time respiration compensation algorithmaccording to an embodiment of the present invention;

FIG. 10 is a flow chart depicting the registration of arespiratory-gated point cloud to a segmented image dataset and thesubsequent deformation of a segmented image dataset to therespiratory-gated point cloud according to an embodiment of the presentinvention;

FIG. 11A illustrates a generated image of an anterior to posterior (A-P)view of a region (or tissue) of interest of a patient and FIG. 11Billustrates a generated image of a lateral view of a region (or tissue)of interest of a patient according to an embodiment of the presentinvention;

FIG. 12 is a perspective view of a generated image of a region (ortissue) of interest of a patient according to an embodiment of thepresent invention;

FIGS. 13A and 13B show an alternative method of generating a 4D datasetfor 4D thoracic registration using surgical instrument navigation systemaccording to an embodiment of the present invention;

FIG. 14 shows an apparatus and method for respiratory 4D dataacquisition and navigation according to an embodiment of the presentinvention;

FIG. 15A is a side perspective view of the implanting of a firstlocalization element and FIG. 15B is a side perspective view of locatingthe first localization element with a second localization elementaccording to an embodiment of the present invention;

FIGS. 16 and 16A show a side perspective view and a cutaway view of asteerable catheter according to an embodiment of the present invention;

FIG. 17A shows a side perspective view of a steerable catheter andassociated possible biopsy devices including a forceps device (FIG.17B), an auger device (FIG. 17E), a boring bit device (FIG. 17C), abrush device (FIG. 17D), an aspiration needle device (FIG. 17F), and aside exiting tip component (FIG. 17G) according to various embodiment ofthe present invention;

FIGS. 18A and 18B are a side views of a steerable catheter deflected byactuating the steering actuator according to an embodiment of thepresent invention;

FIGS. 19 and 19A are a top cutaway view and a cross-section view of anavigated steerable catheter wherein the steerable shaft portioncomprises spline rings according to an embodiment of the presentinvention;

FIGS. 20A, 20B, and 20C show a side, top, and bottom view of the distalend portion of a steerable catheter wherein the biopsy device comprisesan angled or directional pattern visible via fluoroscopic imagingaccording to an embodiment of the present invention;

FIGS. 21A, 21B, and 21C show a side, top, and bottom view of the distalend portion of a steerable catheter wherein the biopsy device comprisesmarkers visible via fluoroscopic imaging according to an embodiment ofthe present invention;

FIGS. 22 and 22A show a side view of the distal end portion of asteerable catheter wherein the biopsy device comprises an echogenicpattern of partially spherical indentations visible via ultrasonicimaging according to an embodiment of the present invention;

FIGS. 23 and 23A show a side view of the distal end portion of asteerable catheter wherein the biopsy device comprises an echogenicpattern that is visible via ultrasonic imaging according to anembodiment of the present invention;

FIG. 24 is a side cutaway view of the distal end portion of a steerablecatheter wherein the biopsy device comprises an actuatablesensor-equipped forceps device according to an embodiment of the presentinvention;

FIGS. 25A and 25B are side cutaway views of the distal end portion of asteerable catheter wherein the biopsy device comprises a navigated augerdevice according to an embodiment of the present invention;

FIGS. 26A and 26B are side cutaway views of the distal end portion of asteerable catheter wherein the biopsy device comprises a non-navigatedauger device according to an embodiment of the present invention;

FIG. 27 is a side view of the distal end portion of a steerable catheterwherein the biopsy device comprises an auger device having wherein thetissue collection region comprises a viewing window according to anembodiment of the present invention;

FIGS. 28A and 28B are side cutaway views of the distal end portion of asteerable catheter wherein the biopsy device comprises a navigatedboring bit device according to an embodiment of the present invention;

FIG. 29 is a side perspective cutaway view of the distal end portion ofa steerable catheter wherein the biopsy device comprises a boring bitdevice according to an embodiment of the present invention;

FIGS. 30A and 30B are side cutaway views of the distal end portion of asteerable catheter wherein the biopsy device comprises a non-navigatedboring bit according to an embodiment of the present invention;

FIGS. 31A and 31B are side cutaway views of the distal end portion of asteerable catheter wherein the biopsy device comprises a navigatedboring bit device wherein the actuation wire is hollow and to which avacuum pressure is applied to assist in the removal of tissue accordingto an embodiment of the present invention;

FIG. 32 is a side perspective cutaway view of the distal end portion ofa steerable catheter wherein the biopsy device comprises a boring bitdevice wherein the actuation wire is hollow and to which a vacuumpressure is applied to assist in the removal of tissue according to anembodiment of the present invention;

FIG. 33 is a side cutaway view of the distal end portion of a steerablecatheter wherein the biopsy device comprises an aspiration needleaccording to an embodiment of the present invention;

FIGS. 34A and 34B show a side view and a side perspective cutaway viewof the distal end portion of a steerable catheter wherein the biopsydevice comprises a brush device according to an embodiment of thepresent invention;

FIG. 35 is a front perspective view of the surgical catheter with a sideexiting medical instrument according to an embodiment of the presentinvention;

FIG. 36 is a side view illustrating the calibration of path of themedical instrument as it extends out the side exit of the surgicalcatheter according to an embodiment of the present invention;

FIG. 37 is a side cutaway view of the surgical catheter wherein themedical instrument may comprise possible devices including a forcepsdevice (FIG. 37A), a boring bit device (FIG. 37B), a brush device (FIG.37C), and an auger device (FIG. 37D) according to various embodiment ofthe present invention;

FIG. 38 is a side cutaway view of the surgical catheter with a medicalinstrument extended through the side exit, wherein the medicalinstrument is an aspiration needle, and a localization element is at thedistal end portion of an elongate flexible shaft according to anembodiment of the present invention;

FIG. 39 is a side cutaway view of the surgical catheter with a medicalinstrument extended through the side exit and a localization element isat the tip of the side exiting tip component according to an embodimentof the present invention;

FIG. 40 is a side cutaway view of the surgical catheter wherein themedical instrument comprises a shape memory alloy extended out of theside exit according to an embodiment of the present invention;

FIGS. 41A-41D are side views of the medical instrument in the extendedposition disposed at various angles relative to the longitudinal axis ofan elongate flexible shaft at a side exit according to an embodiment ofthe present invention;

FIG. 41E is a side view of the medical instrument in the extendedposition disposed at an angle relative to the longitudinal axis of anelongate flexible shaft at a side exit wherein an arc is introduced intothe elongate flexible shaft according to an embodiment of the presentinvention;

FIGS. 42A, 42B, and 42C are a side, top, and bottom view of the surgicalcatheter comprising an angled or directional pattern visible viafluoroscopic imaging according to an embodiment of the presentinvention;

FIGS. 43A, 43B, and 43C are a side, top, and bottom view of the surgicalcatheter comprising markers visible via fluoroscopic imaging accordingto an embodiment of the present invention;

FIGS. 44A, 44B, and 44C are a side, top, and bottom view of the surgicalcatheter comprising rings visible via fluoroscopic imaging according toan embodiment of the present invention;

FIGS. 45 and 45A are a side view, and detailed partially cut-away viewof the surgical catheter comprising an echogenic pattern visible viaultrasonic imaging according to an embodiment of the present invention;

FIGS. 46 and 46A are a side view, and detailed partially cut-away viewof the surgical catheter comprising an echogenic pattern of partiallyspherical indentations visible via ultrasonic imaging according to anembodiment of the present invention;

FIG. 47 is a side cutaway view of the surgical catheter comprising anechogenic pattern visible via ultrasonic imaging that is on the medicalinstrument according to an embodiment of the present invention;

FIG. 48 is a side cutaway view of the surgical catheter with a medicalinstrument extended through the side exit and a localization element inthe distal end portion of the elongate flexible shaft according to anembodiment of the present invention;

FIGS. 49A, 49B, and 49C show a side, top, and bottom view of thesurgical catheter comprising an angled or directional pattern visiblevia fluoroscopic imaging according to an embodiment of the presentinvention.

FIGS. 50A and 50B are side views of a port offset device according to anembodiment of the present invention;

FIGS. 51A and 51B are side views of a port offset device according to anembodiment of the present invention;

FIGS. 52A, 52B, 52C, and 52D are perspective views of affixinglocalization elements to existing surgical instruments according to anembodiment of the present invention;

DETAILED DESCRIPTION

Referring now to FIG. 1, a surgical instrument navigation system 10 inaccordance with one embodiment of the present invention is operable tovisually simulate a virtual volumetric scene within the body of apatient, such as an internal body cavity, from a point of view of asurgical instrument 12 residing in the cavity of a patient 13. Thesurgical instrument navigation system 10 comprises a surgical instrument12, a processor 16 having a display 18, and a tracking subsystem 20. Thesurgical instrument navigation system 10 may further include (or isaccompanied by) an imaging device 14 that is operable to provide imagedata to the system.

Imaging device 14 can be used to capture images or data of patient 13.Imaging device 14 can be, for example, a computed tomography (CT) device(e.g., respiratory-gated CT device, ECG-gated CT device), a magneticresonance imaging (MRI) device (e.g., respiratory-gated MRI device,ECG-gated MRI device), an X-ray device, or any other suitable medicalimaging device. In one embodiment, imaging device 14 is a computedtomography—positron emission tomography device that produces a fusedcomputed tomography—positron emission tomography image dataset. Imagingdevice 14 is in communication with processor 16 and can send, transfer,copy and/or provide image data taken (captured) of patient 13 toprocessor 16.

Processor 16 includes a processor-readable medium storing coderepresenting instructions to cause processor 16 to perform a process.Processor 16 may be, for example, a commercially available personalcomputer, or a less complex computing or processing device that isdedicated to performing one or more specific tasks. For example,processor 16 may be a terminal dedicated to providing an interactivegraphical user interface (GUI). Alternatively, processor 16 may be acommercially available microprocessor, an application-specificintegrated circuit (ASIC) or a combination of ASICs, which are designedto achieve one or more specific functions, or enable one or morespecific devices or applications. In yet another embodiment, processor16 may be an analog or digital circuit, or a combination of multiplecircuits.

Processor 16 preferably includes a memory component (not shown)comprising one or more types of memory devices. For example, the memorycomponent may comprise a read only memory (ROM) device and/or a randomaccess memory (RAM) device. The memory component may also comprise othertypes of memory devices that may be suitable for storing data in a formretrievable by processor 16. For example, the memory component maycomprise electronically programmable read only memory (EPROM), erasableelectronically programmable read only memory (EEPROM), flash memory, aswell as other suitable forms of memory. The memory component may alsocomprise a non-transitory processor-readable medium. Processor 16 mayalso include a variety of other components, such as for example,coprocessors, graphic processors, etc., depending upon the desiredfunctionality of the code. Processor 16 can store data in or retrievedata from the memory component.

Processor 16 may also comprise components to communicate with devicesexternal to processor 16 by way of an input/output (I/O) component (notshown). According to one or more embodiments of the invention, the I/Ocomponent can include a variety of suitable communication interfaces.For example, the I/O component can include wired connections, such asstandard serial ports, parallel ports, universal serial bus (USB) ports,S-video ports, local area network (LAN) ports, and small computer systeminterface (SCSI) ports. Additionally, the I/O component may include, forexample, wireless connections, such as infrared ports, optical ports,Bluetooth® wireless ports, wireless LAN ports, or the like.

In one embodiment, processor 16 is connected to a network (not shown),which may be any form of interconnecting network including an intranet,such as a local or wide area network, or an extranet, such as the WorldWide Web or the Internet. The network can be physically implemented on awireless or wired network, on leased or dedicated lines, including avirtual private network (VPN).

In one embodiment, processor 16 can receive image data from imagingdevice 14 and generate a segmented image dataset using varioussegmentation techniques, such as Hounsfield unit thresholding,convolution, connected component, or other combinatory image processingand segmentation techniques. For example, in one embodiment processor 16can determine a distance and direction between the position of any twodata points within a respiratory-gated point cloud (as described ingreater detail elsewhere herein) during multiple instants in time, andstore the image data, as well as the position and distance data, withinthe memory component. Multiple images can be produced providing a visualimage at multiple instants in time through the path of motion of thepatient's body.

Surgical instrument 12 may be any medical device used in a medicalprocedure. In one embodiment, surgical instrument 12 comprises arelatively flexible catheter that may be guided to the region or tissueof interest. Thus, for example, surgical instrument 12 may comprise orbe used to implant one or more surgical devices such as a guide wire, apointer probe, a stent, a seed, an implant, or an endoscope. It is alsoenvisioned that the surgical instruments may encompass medical deviceswhich are used for exploratory purposes, testing purposes or other typesof medical procedures. Additionally or alternatively, surgicalinstrument 12 may incorporate one or more localization elements 24 thatare detectable by tracking subsystem 20. As illustrated in FIG. 10,surgical instrument 12 is connected by wire to tracking subsystem 20; inalternative embodiments, surgical instrument 12 may be wirelesslyconnected to tracking subsystem 20.

Imaging device 14 may be used to capture volumetric scan data (see box32 of FIG. 2) representative of an internal region of interest withinpatient 13. The scan data, preferably three-dimensional data, may beobtained prior to and/or during surgery on patient 13 and stored in thememory component associated with processor 16. It should be understoodthat volumetric scan data may be acquired using various known medicalimaging devices 14, including but not limited to a magnetic resonanceimaging (MRI) device, a computed tomography (CT) imaging device, apositron emission tomography (PET) imaging device, a 2D or 3Dfluoroscopic imaging device, and 2D, 3D or 4D ultrasound imagingdevices. In the case of a two-dimensional ultrasound imaging device orother two-dimensional image acquisition device, a series oftwo-dimensional data sets may be acquired and then assembled intovolumetric data as is well known in the art using a two-dimensional tothree-dimensional conversion.

Dynamic reference frame 19 may be attached to patient 13 proximate tothe region (tissue) of interest within the patient 13. For ease ofillustration, dynamic reference frame 19 is attached to the forehead ofpatient 13 in FIG. 1; in an actual medical procedure, dynamic referenceframe 19 may be located in a cavity, vessel or otherwise within patient13. In one embodiment, dynamic reference frame 19 includes localizationelements detectable by the tracking subsystem 20 to enable dynamicreference frame 19 to function as a point of reference for trackingsubsystem 20 during the procedure as further described below.

Tracking subsystem 20 is also configured to track localization elements24 associated with surgical instrument 12. In general, trackingsubsystem 20 may comprise any tracking system typically employed inimage guided surgery, including but not limited to an electromagnetictracking system. An example of a suitable electromagnetic trackingsubsystem is the AURORA electromagnetic tracking system, commerciallyavailable from Northern Digital Inc. in Waterloo, Ontario Canada. In oneembodiment, tracking subsystem 20 is an electromagnetic tracking system,typically comprising an electromagnetic field generator 22 that emits aseries of electromagnetic fields designed to engulf patient 13, andlocalization elements 24 coupled to surgical instrument 12 could becoils that would receive an induced voltage that could be monitored andtranslated into a coordinate position of localization elements 24. Incertain embodiments, localization element 24 may be electrically coupledto twisted pair conductors to provide electromagnetic shielding of theconductors. This shielding prevents voltage induction along theconductors when exposed to the magnetic flux produced by theelectromagnetic field generator. The twisted pair conductors extend fromthe localization element through surgical instrument 12.

FIG. 2 illustrates a flowchart of a technique for simulating a virtualvolumetric scene of a body cavity from a point of view of a surgicalinstrument positioned within the patient. Volumetric scan data capturedby imaging device 14 (see box 32) may be registered to patient 13 (seebox 34) using dynamic reference frame 19. This registration process issometimes referred to as registering image space to patient space.Often, the volumetric scan data captured by imaging device 14 is alsoregistered to other image datasets, typically an image dataset acquiredat an earlier point time or an atlas. Registration of the image space topatient space is accomplished through knowledge of the coordinatevectors of at least three non-collinear points in the image space andthe patient space. FIG. 3A, for example, illustrates an initiallygenerated point cloud superimposed on a segmented image dataset of apatient's respiratory system and FIG. 3B illustrates a point cloudregistered to a segmented image dataset of a patient's respiratorysystem.

Registration of image space to patient space for image guided surgery(see box 34 of FIG. 2) can be completed by different known techniques.Registration can be performed in multiple ways: point registration,pathway registration, 2D/3D image registration, etc. Such registrationmay be performed using 4D-gating information to track the patient'smotion such as respiration and/or heartbeat. For example, point-to-pointregistration may be accomplished by identifying points in an image spaceand then touching the same points in patient space. These points aregenerally anatomical landmarks that are easily identifiable on thepatient. By way of further example, surface registration involves theuser's generation of a surface in patient space by either selectingmultiple points or scanning, and then accepting the best fit to thatsurface in image space by iteratively calculating with the processoruntil a surface match is identified. By way of further example, repeatfixation devices entail the user repeatedly removing and replacing adevice (i.e., dynamic reference frame, etc.) in known relation to thepatient or image fiducials of the patient. By way of further example,automatic registration is accomplished by first attaching the dynamicreference frame to the patient prior to acquiring image data. It isenvisioned that other known registration procedures are also within thescope of the present invention, such as that disclosed in U.S. Pat. No.6,470,207, which is hereby incorporated by reference in its entirety.Once registration is complete the system can determine potential regionsof the image dataset that may not match the patient's real-time anatomy.After image registration (see box 34 of FIG. 2) the image data may berendered (see box 36 of FIG. 2) as a volumetric perspective image and/ora surface rendered image of the region of interest based on the scandata using rendering techniques well known in the art.

During surgery, surgical instrument 12 is directed by the physician orother healthcare professional to the region (or tissue) of interestwithin patient 13. Tracking subsystem 20 preferably employselectromagnetic sensing to capture position data (see box 37 of FIG. 2)indicative of the location and/or orientation of surgical instrument 12within patient 13. Tracking subsystem 20 may be defined as anelectromagnetic field generator 22 and one or more localization elements24 (e.g., electromagnetic sensors) may be integrated into the items ofinterest, such as the surgical instrument 12. In one embodiment,electromagnetic field generator 22 may be comprised of three or morefield generators (transmitters) mounted at known locations on a planesurface and localization elements (receivers) 24 are further defined asa single coil of wire. The positioning of the field generators(transmitter) and the localization elements (receivers) may also bereversed, such that the generators are associated with surgicalinstrument 12 and the receivers are positioned elsewhere. Although notlimited thereto, electromagnetic field generator 22 may be affixed to anunderneath side of the operating table that supports the patient.

In certain embodiments, localization element 24 comprises a six (6)degree of freedom (6DOF) electromagnetic sensor. In other embodiments,localization element 24 comprises a five (5) degree of freedom (5DOF)electromagnetic sensor. In other embodiments, localization element 24comprises other localization devices such as radiopaque markers that arevisible via fluoroscopic imaging and echogenic patterns that are visiblevia ultrasonic imaging. In yet other embodiments, localization elements24 can be, for example, infrared light emitting diodes, and/or opticalpassive reflective markers. Localization elements 24 can also be, or beintegrated with, one or more fiber optic localization (FDL) devices. Inother embodiments surgical instrument 12 is non-navigated, such that itdoes not include any localization elements.

In operation, the field generators of localization device 22 generatemagnetic fields which are detected by localization element 24. Bymeasuring the magnetic field generated by each field generator atlocalization element 24, the location and orientation of localizationelement 24 may be computed, thereby determining position data forlocalization element 24 associated with surgical instrument 12. Althoughnot limited thereto, exemplary electromagnetic tracking subsystems arefurther described in U.S. Pat. Nos. 5,913,820; 5,592,939; and 6,374,134which are incorporated herein by reference in their entirety. Inaddition, it is envisioned that other types of position tracking devicesare also within the scope of the present invention. For instance,tracking subsystem 20 may comprise a non-line-of-sight device based onsonic emissions or radio frequency emissions. In another instance, arigid surgical instrument, such as a rigid endoscope may be trackedusing a line-of-sight optical-based tracking subsystem (i.e., LED's,passive markers, reflective markers, etc.).

Position data for localization element 24, such as location and/ororientation data from the tracking subsystem 20 is in turn relayed tothe processor 16. Processor 16 is adapted to receiveposition/orientation data (see box 37 of FIG. 2) from tracking subsystem20 and the volumetric perspective and/or surface image data may befurther manipulated (see box 38 of FIG. 2) based on theposition/orientation data for surgical instrument 12 received fromtracking subsystem 20. Specifically, the volumetric perspective orsurface rendered image is rendered from a point of view which relates toposition of the surgical instrument 12. For instance, at least onelocalization element 24 may be positioned at the distal end of surgicalinstrument 12, such that the image is rendered from a leading point onthe surgical instrument. In this way, surgical instrument navigationsystem 10 of the present invention is able, for example, to visuallysimulate a virtual volumetric scene of an internal cavity from the pointof view of surgical instrument 12 residing in the cavity without the useof an endoscope. It is readily understood that tracking two or morelocalization elements 24 which are embedded in surgical instrument 12enables orientation of surgical instrument 12 to be determined by thesystem 10.

As surgical instrument 12 is moved by the physician or other healthcareprofessional within the region of interest, its position and orientationmay be tracked and reported on a real-time basis by tracking subsystem20. Referring again to FIG. 2, the volumetric perspective image may thenbe updated by manipulating (see box 38) the rendered image data (see box36) based on the position of surgical instrument 12. The manipulatedvolumetric perspective image (see box 38) is displayed 40 as a primaryimage on a display device 18 associated with the processor 16. Thedisplay 18 is preferably located such that it can be easily viewed bythe physician or other healthcare professional during the medicalprocedure. In one embodiment, the display 18 may be further defined as aheads-up display or any other appropriate display. The image may also bestored by processor 16 for later playback, should this be desired.

It is envisioned that the primary of the region of interest may besupplemented by other secondary images. For instance, known imageprocessing techniques may be employed to generate various multi-planarimages of the region of interest. Alternatively, images may be generatedfrom different view points (see box 39) as specified by a physician orother healthcare professional, including views from outside of thevessel or cavity or views that enable the user to see through the wallsof the vessel using different shading or opacity. In another instance,the location data of the surgical instrument may be saved and playedback in a movie format. It is envisioned that these various secondaryimages may be displayed simultaneously with or in place of the primaryperspective image.

In addition, surgical instrument 12 may be used to generate real-timemaps corresponding to an internal path traveled by the surgicalinstrument or an external boundary of an internal cavity. Real-time mapsmay be generated by continuously recording the position of theinstrument's localized tip and its full extent. A real-time map may begenerated by the outermost extent of the instrument's position andminimum extrapolated curvature as is known in the art. The map may becontinuously updated as the instrument is moved within the patient,thereby creating a path or a volume representing the internal boundaryof the cavity. It is envisioned that the map may be displayed in a wireframe form, as a shaded surface or other three-dimensional computerdisplay modality independent from or superimposed on the volumetricperspective image of the region of interest. It is further envisionedthat the map may include data collected from a localization elementembedded into the surgical instrument, such as pressure data,temperature data or electro-physiological data. In this case, the mapmay be coded with a color or some other visual indicia to represent thecollected data.

FIG. 4 illustrates another type of secondary image 28 which may bedisplayed in conjunction with the primary perspective image 30. In thisinstance, primary perspective image 30 is an interior view of an airpassage within patient 13. Secondary image 28 is an exterior view of theair passage which includes an indicia or graphical representation 29that corresponds to the location of surgical instrument 12 within theair passage. In FIG. 4, indicia 29 is shown as a crosshairs. It isenvisioned that other indicia may be used to signify the location of thesurgical instrument in the secondary image. As further described below,secondary image 28 may be constructed by superimposing indicia 29 ofsurgical instrument 12 onto the manipulated image data 38.

The displayed indicia 29 of surgical instrument 12 tracks the movementof surgical instrument 12 as it is moved by the physician or otherhealthcare professional within patient 13. In certain instances, thecardiac or respiration cycle of the patient may cause surgicalinstrument 12 to flutter or jitter within the patient. For instance, asurgical instrument 12 positioned in or near a chamber of the heart willmove in relation to the patient's heart beat. In this instance, theindicia of the surgical instrument 12 will likewise flutter or jitter onthe displayed image (see box 40 of FIG. 2). It is envisioned that otheranatomical functions which may affect the position of the surgicalinstrument 12 within the patient are also within the scope of thepresent invention. Rather than display indicia 29 of surgical instrument12 on a real-time basis, the display of indicia 29 of surgicalinstrument 12 is periodically updated based on a timing signal fromtiming signal generator 26. In one exemplary embodiment, the timingsignal generator 26 is electrically connected to tracking subsystem 20.

As shown by the flowchart of FIG. 5, another embodiment may be atechnique for synchronizing the display of an indicia or graphicalrepresentation of surgical instrument 12 with cardiac or respirationcycle of the patient in order to reduce flutter. The display of anindicia of surgical instrument 12 may be synchronized with an anatomicalfunction, such as the cardiac or respiration cycle of the patient. Asdescribed above, imaging device 14 may be used to capture (see box 32 ofFIG. 5) volumetric scan data representative of an internal region ofinterest within a given patient. An may then be rendered (see box 36 ofFIG. 5) from the volumetric scan data by processor 16. A timing signalgenerator 26 (not shown) may be operable to generate and transmit atiming signal (see box 46 of FIG. 5) that correlates to at least one of(or both) the cardiac cycle or the respiration cycle of patient 13. Fora patient having a consistent rhythmic cycle, the timing signal might bein the form of a periodic clock signal. Alternatively, the timing signalmay be derived from an electrocardiogram signal from the patient 13. Oneskilled in the art will readily recognize other techniques for derivinga timing signal that correlate to at least one of the cardiac orrespiration cycle or other anatomical cycle of the patient. Theacquisition of position data (see box 37 of FIG. 5) for surgicalinstrument 12 may then be synchronized to the timing signal.

Tracking subsystem 20 is, in turn, operable to report position data (seebox 37 of FIG. 5) for surgical instrument 12 in response to a generatedtiming signal (see box 46 of FIG. 5) received from timing signalgenerator 26. The position of indicia 29 of surgical instrument 12 maythen be updated and superimposed (see box 50 of FIG. 5) on the displayof the image data (see box 40 of FIG. 5). It is readily understood thatother techniques for synchronizing the display of indicia 29 of surgicalinstrument 12 based on the timing signal are within the scope of thepresent invention, thereby eliminating any flutter or jitter which mayappear on the displayed image (see box 40 of FIG. 5). It is alsoenvisioned that a path (or projected path) of surgical instrument 12 mayalso be illustrated on displayed image data (see box 40 of FIG. 5).

In another aspect of the present invention, surgical instrumentnavigation system 10 may be further adapted to display four-dimensionalimage data for a region of interest as shown in the flowchart of FIG. 6.In this case, imaging device 14 is operable to capture 4D volumetricscan data (see box 62) for an internal region of interest over a periodof time, such that the region of interest includes motion that is causedby either the cardiac cycle or the respiration cycle of patient 13. Avolumetric perspective image of the region may be rendered (see box 64)from the captured 4D volumetric scan data (see box 62) by processor 16as described above. The four-dimensional image data may be furthersupplemented with other patient data, such as temperature or bloodpressure, using a color code or some other visual indicia.

The display of the volumetric perspective image may be synchronized (seebox 66) in real-time with the cardiac or respiration cycle of patient 13by adapting processor 16 to receive a generated timing signal (see box46) from timing signal generator 26. As described above, the timingsignal generator 26 is operable to generate and transmit a timing signalthat correlates to either the cardiac cycle or the respiration cycle ofpatient 13. In this way, the 4D volumetric perspective image may besynchronized (see box 66) with the cardiac or respiration cycle ofpatient 13. The synchronized image is then displayed (see box 68) on thedisplay 18 of the system. The four-dimensional synchronized image may beeither (or both of) the primary image rendered from the point of view ofthe surgical instrument or the secondary image depicting the indicia ofthe position of surgical instrument 12 within patient 13. It is readilyunderstood that the synchronization process is also applicable totwo-dimensional image data acquire over time.

To enhance visualization and refine accuracy of the displayed imagedata, the surgical navigation system can use prior knowledge such as asegmented vessel or airway structure to compensate for error in thetracking subsystem or for inaccuracies caused by an anatomical shiftoccurring since acquisition of scan data. For instance, it is known thatsurgical instrument 12 being localized is located within a given vesselor airway and, therefore may be displayed within the vessel or airway.Statistical methods can be used to determine the most likely location;within the vessel or airway with respect to the reported location andthen compensate so the display accurately represents surgical instrument12 within the center of the vessel or airway. The center of the vesselor airway can be found by segmenting the vessels or airways from thethree-dimensional datasets and using commonly known imaging techniquesto define the centerline of the vessel or airway tree. Statisticalmethods may also be used to determine if surgical instrument 12 haspotentially punctured the vessel or airway. This can be done bydetermining the reported location is too far from the centerline or thetrajectory of the path traveled is greater than a certain angle (worsecase 90 degrees) with respect to the vessel or airway. Reporting thistype of trajectory (error) may be desired by the physicians or otherhealthcare professionals. The tracking along the center of the vessel orairway may also be further refined by correcting for motion of therespiratory or cardiac cycle, as described above. While navigating alongthe vessel or airway tree, prior knowledge about the last known locationcan be used to aid in determining the new location. Surgical instrument12 or other navigated device follows a pre-defined vessel or airway treeand therefore cannot jump from one branch to the other without travelingalong a path that would be allowed. The orientation of surgicalinstrument 12 or other navigated device can also be used to select themost likely pathway that is being traversed. The orientation informationcan be used to increase the probability or weight for selected locationor to exclude potential pathways and therefore enhance system accuracy.

Surgical instrument navigation system 10 of the present invention mayalso incorporate atlas maps. It is envisioned that three-dimensional orfour-dimensional atlas maps may be registered with patient specific scandata or generic anatomical models. Atlas maps may contain kinematicinformation (e.g., heart models) that can be synchronized withfour-dimensional image data, thereby supplementing the real-timeinformation. In addition, the kinematic information may be combined withlocalization information from several instruments to provide a completefour-dimensional model of organ motion. The atlas maps may also be usedto localize bones or soft tissue which can assist in determiningplacement and location of implants.

In general, a consistent feature between lung scans is the existence ofan airway tree within the lung tissue, consisting of multiple branchesand carinas. The branches and carinas, however, move as a consequence ofa patient's respiration. To provide more accurate navigation of aninstrument through the airway tree of a patient, a set of data pointsmay be collected from a patient pathway (e.g., an airway) and a model ofpoints may be calculated to match the image dataset. In one embodiment,each discrete segment of the image dataset and its correspondinginformation are matched to the collected points, creating a “pointcloud” of information. Then, the data points that create the outerregion or shell of the point cloud are determined, followed bycorrelation or matching of the outer points to the patient's 3D imagedata sets.

A respiratory-gated point cloud comprises a plurality of data pointscorresponding to the internal volume of a patient's respiratory systemmeasured by a localization element during the respiration cycle of apatient. Each data point of the respiratory-gated point cloud comprisesthree dimensional data (x, y, and z location) in reference to a 3Dcoordinate system. In this embodiment, each data point of therespiratory-gated point cloud may be gated to the respiration cycle ofthe patient. The respiratory-gated point cloud also comprises a fourthdimension representing the phase (inspiration, expiration, and, ifdesired, points in between) of the respiration cycle of the patient atwhich point the individual data point was generated. The phaseinformation may be provided by a patient tracker that real-time tracksthe patient's respiratory cycle. In certain embodiments, the generationof the individual data points in the point cloud may occur on atime-gated basis triggered by a physiological signal of the patient'srespiration cycle. In other embodiments, a respiratory-gated point cloudcan be collected at inspiration and another respiratory-gated pointcloud collected at expiration. These two respiratory-gated point cloudscan then be matched to an image dataset to assist registration.Alternatively, a single respiratory-gated point cloud can be collectedincluding data points from both inspiration and expiration and matchedto an image dataset.

Referring now to FIG. 7, a respiratory-gated point cloud can becollected in one embodiment using a surgical instrument navigationsystem 10 (not shown) comprising catheter 612 having localizationelement 624 at the distal end thereof. More specifically, a physician orother healthcare professional moves catheter 612 through a plurality oflocations 614 within the branches of a patient's respiratory system 602,including trachea 608, right main bronchus (RMB) 604, and left mainbronchus (LMB) 606 over a full respiration cycle of patient 613 to formrespiratory-gated point cloud corresponding to position/orientation dataof localization element 624. According to one particular embodiment, arespiratory signal is used to gate the localization information.Additionally or alternatively, the respiratory signal can be derived,for example, using a device that records the resistance between twolocations on the patient; such a method is similar to a variablepotentiometer in that the resistance of the patient changes between twofixed points as the patient inhales and exhales. Thus, the resistancecan be measured to create a respiratory signal. When a signal indicatinga particular phase of the respiration cycle is received, the processorbegins acquiring valid position/orientation data regarding thelocalization element(s) 624 in catheter 612 through a plurality oflocations 614 within the branches of a patient's respiratory system 602,thereby generating respiratory-gated point cloud. Once the respirationcycle moves outside of that phase, a stop signal halts the point clouddata collection. In this way, it is not necessary to track the motion ofthe patient's anatomy if the respiratory motion is the only motionoccurring.

Density filtering of the generated respiratory-gated point clouds canreduce the number of duplicate data points generated which cansignificantly decrease the processing time. Depending on the desiredstrength of filtering, a duplicate data point in the respiratory-gatedpoint is defined as having identical three dimensional coordinates (x,y, and z) to another data point in the respiratory-gated point cloudwherein both points were generated in the same respiratory phase or aduplicate data point in the respiratory-gated point is defined as havingthree dimensional coordinates x1, y1, and z1 within a certain distanceto another data point in the respiratory-gated point cloud having threedimensional coordinates x2, y2, and z2 wherein both points weregenerated in the same respiratory phase. This duplicated data point, andany additional duplicate data points can be eliminated, leaving only onedata point for each three dimensional coordinate and correspondingrespiratory phase. In another embodiment, additional density filteringcan be done by eliminating duplicate data points without reference to agiven respiratory phase. This would eliminate duplicate data points fromthe respiratory-gated point cloud that were generated throughoutmultiple phases. By eliminating the duplicate data points, a processorneed not perform subsequent calculations of unnecessary data points.

Additionally, in certain embodiments, the generated point cloud may becompared to the segmented image data to determine the strength orweighting of each point collected in the point cloud. Calculating thestrength or weighting of discrete points in the point cloud can enhanceregistration accuracy. By way of example, collecting a single string ofpoints that are only 1 mm wide to represent an airway that is 5-6 mmwide as determined in the image model would be an insufficient pointcloud. Feedback can be provided to the user such as color coding or someother visual indicia to identify the strength of the point cloud.

In one exemplary embodiment, a physician or other healthcareprofessional captures a respiratory-gated point cloud and the capturedcloud is density filtered as previously described to form adensity-filtered point cloud comprising unclassified point cloud datapoints. The density-filtered point cloud may then be classified using afirst k-means algorithm which performs orientation classificationresulting in the data points in the respiratory-gated point cloud beingclassified into the trachea, the right main bronchus and the left mainbronchus. A second k-means algorithm is performed to further classifythe data points in the respiratory-gated point cloud into controlpoints. The respiratory-gated point cloud may then be registered to apre-existing image dataset and the data points of the respiratory-gatedpoint cloud are weighted. Each data point in the respiratory-gated pointcloud may then be displayed to the user with a color code or some othervisual indicia corresponding to the calculated weight for each datapoint in the respiratory-gated point cloud. In certain embodiments,feedback may be provided to the physician or other healthcareprofessional indicating that additional respiratory-gated point clouddata points may be collected in locations having lesser weighting. Thismethod may then be repeated until a desired weighting is achieved acrossthe respiratory-gated point cloud.

Image datasets may not perfectly match if the image data was acquired ata different phase in the respiration cycle (e.g., full inspiration,partial inspiration, full expiration, etc.) or if the patient's anatomyhas been changed due to positioning on the table, weight gain/loss, skinshift, delivery of drugs, etc. In such embodiments, an image datasettaken at a first time point can be modified or deformed to bettercorrespond to the respiratory-gated point cloud generated during themedical procedure (i.e., a second and subsequent time point).Additionally, a sequence of motion of the respiratory-gated point cloudcan be generated over the complete procedure or significant period oftime. The distance, range, acceleration, and speed between one or moreselected pairs of respiratory-gated data points within the point cloudgenerated by the localization element 624 (see FIG. 7) can be determinedand various algorithms can be used to analyze and compare the distancebetween selected data points at given instants in time.

Referring now to FIG. 8, a method for modifying or deforming a segmentedimage dataset for a region of a respiratory system of a patient to thecorresponding anatomy of a patient's respiratory system in oneembodiment comprises forming (see box 700) a respiratory-gated pointcloud of data that demarcates anatomical features in a region of apatient's respiratory system at one or more discrete phases within arespiration cycle of a patient. The respiratory-gated point cloud isthen density filtered (see box 702). The density filtered point cloud isthen classified (see box 703) according to anatomical points ofreference in a segmented image dataset for the region of the patient'srespiratory system (as described above), and a segmented image datasetfor the region of the respiratory system is modified (or deformed) (seebox 704) using a deformation vector field to correspond to theclassified anatomical points of reference in the density filteredrespiratory-gated point cloud. In certain embodiments, the phases atwhich the respiratory-gated point cloud is formed include inspiration,expiration and phases in between.

A deformation vector field can be calculated between a first set ofpoints in the respiratory-gated point cloud that correspond toinspiration and a second set of points in the respiratory-gated pointcloud that correspond to expiration. This deformation vector field maythen be used to modify or deform a pre-existing or pre-acquiredsegmented image dataset, taken from a first time interval, to correspondto the correlated anatomical points of reference in therespiratory-gated point cloud, taken during a second time interval. Incertain embodiments, the segmented image dataset to be modified is froma first discrete phase of the patient's respiration cycle and therespiratory-gated point cloud is from a second and different discretephase of the patient's respiration cycle. Accordingly, a pre-existing orpre-acquired segmented image dataset can be from an inspiration phaseand it can be modified or deformed to the expiration phase using thedeformation vector field calculated from the respiratory-gated pointcloud. Thus a segmented image dataset need not require an image for eachphase of the patient's respiration cycle.

A deformation vector field can be calculated between data points in therespiratory-gated point cloud that correspond to different phases of thepatient's respiratory cycle. The image dataset from a first timeinterval may then be modified or deformed by the deformation vectorfield to match the anatomy of the patient during the second timeinterval. This modification or deformation process can be donecontinuously during the medical procedure, producing simulatedreal-time, intra-procedural images illustrating the orientation andshape of the targeted anatomy as a catheter, sheath, needle, forceps,guidewire, fiducial delivery devices, therapy device (ablation modeling,drug diffusion modeling, etc.), or similar structure(s) is/are navigatedto the targeted anatomy. Thus, during the medical procedure, thephysician or other healthcare professional can view selected modified ordeformed image(s) of the targeted anatomy that correspond to andsimulate real-time movement of the anatomy. In addition, during amedical procedure being performed during the second time interval, suchas navigating a catheter or other instrument or component thereof to atargeted anatomy, the location(s) of a localization element (e.g., anelectromagnetic coil sensor) coupled to the catheter during the secondtime interval can be superimposed on an image of a catheter. Thesuperimposed image(s) of the catheter can then be superimposed on themodified or deformed image(s) from the first time interval, providingsimulated real-time images of the catheter location relative to thetargeted anatomy. This process and other related methods are describedin U.S. Pat. No. 7,398,116, the entire disclosure of which isincorporated herein by reference.

The deformation vector field may be calculated between a first set ofpoints in the respiratory-gated point cloud that correspond to a firstrespiration phase and a second set of points in the respiratory-gatedpoint cloud that correspond to a second respiration phase. Typically,the first respiration phase is inspiration and the second respirationphase is expiration. Additionally, the two phases can be reversedwherein the first phase is expiration and the second phase isinspiration. For example, the deformation vector field can be applied tomodify or deform an image dataset of 3D fluoroscopic images or CT imagesin order to compensate for different patient orientations, patientposition, respiration, deformation induced by the catheter or otherinstrument, and/or other changes or perturbations that occur due totherapy delivery or resection or ablation of tissue.

In some embodiments, for example, real-time respiration compensation canbe determined by applying an inspiration-to-expiration deformationvector field. In combination with the respiratory signal, for example,the surgical instrument location can be calculated using the deformationvector field. A real-time surgical instrument tip correction vector canbe applied to a 3D localized instrument tip. The real-time correctionvector is computed by scaling an inspiration-to-expiration deformationvector (found from the inspiration-to-expiration deformation vectorfield) based on the respiratory-gated point cloud. This correctionvector can then be applied to the 3D localized surgical instrument tip.This can further optimize accuracy during navigation.

An example of an algorithm for real-time respiration compensation can befound in FIG. 9. In accordance with this algorithm, for each 3Dlocalized point

:

(a) find v_(i) such that scalar d is minimized;

(b) compute c, wherein:c=−v _(i) t

and (c) compute

, wherein:

=

+cThus,

is a respiration compensated version of

.

Although FIG. 9 and the above discussion generally relate to real-timerespiration motion, it will be understood that these calculations anddeterminations may also be applied to real-time heartbeat and/or vesselmotion compensation, or any other motion of a dynamic body (e.g., thepatient's body, an organ, or tissue thereof) as described herein. In oneembodiment, for example, the deformation vector field is calculatedbased upon inspiration and expiration. In another embodiment, forexample, the deformation vector field is calculated based uponheartbeat. In yet another embodiment, for example, the deformationvector field is based upon vessel motion. In these and otherembodiments, it is also possible to extend these calculations anddeterminations to develop multiple deformation vector fields acrossmultiple patient datasets, by acquiring the multiple datasets over thecourse of, for example, a single heartbeat cycle or a single respirationcycle.

Deformation of 2D images can also be calculated based upon therapeuticchange of tissue, changes in Hounsfield units for images, patient motioncompensation during the imaging sequence, therapy monitoring, andtemperature monitoring with fluoroscopic imaging, among other things.One potential issue with conventional therapy delivery, for instance, ismonitoring the therapy for temperature or tissue changes. In accordancewith the methods described herein, this monitoring can be carried outusing intermittent fluoroscopic imaging, where the images arecompensated between acquisition times to show very small changes inimage density, which can represent temperature changes or tissue changesas a result of the therapy and/or navigation.

Another method to modify/deform the image dataset and match to thepatient is to segment the airway from the image dataset and skeletonizeit to find the central airway tree. The physician or other healthcareprofessional can then modify/deform the image dataset by identifyingpoints within the image space and patient space that match such as themain carina and/or collect multiple branch information that definesbranches and carina points between branches. These points can be used tomodify or deform the image dataset. Deformation of a complete 3D volumewould be time consuming so methods to create deformation matrices forregions may be preferred.

In general, the embodiments described herein have applicability in“Inspiration to Expiration”-type CT scan fusion. According to variousmethods, the user navigates on the expiration CT scan to aid accuracy,while using the inspiration scan to aid airway segmentation. In oneembodiment, for example, a user could complete planning and pathwaysegmentation on an inspiration scan of the patient. Preferably, adeformation vector field is created between at least two datasets. Thedeformation vector field may then be applied to the segmented vesselsand/or airways and the user's planned path and target. In these andother embodiments, the deformation vector field can also be applied tomultiple datasets or in a progressive way to create a moving underlyingdataset that matches the patient's respiratory or cardiac motion. Inother embodiments, using a respiratory-gated point cloud, a deformationvector field is calculated between a first set of points in therespiratory-gated point cloud that correspond to inspiration and asecond set of points in the respiratory-gated point cloud thatcorrespond to expiration. This deformation vector field may then used tomodify or deform a pre-existing or pre-acquired segmented image datasetto correspond to the correlated anatomical points of reference in therespiratory-gated point cloud.

In accordance with various embodiments, “Inspiration to Expiration” CTfusion using the lung lobe centroid and vector change to modify anairway model may be used to translate and scale each airway based on thelung lobe change between scans. The lung is constructed of multiplelobes and these lobes are commonly analyzed for volume, shape, andtranslation change. Each lobe changes in a very different way during thepatient's respiration cycle. Using this information to scale andtranslate the airways that are located in each lobe, it is possible toadapt for airway movement. This scaled airway model can then be linkedto the 4D tracking of the patient as described herein. In accordancewith various embodiments using a respiratory-gated point cloud, thistechnique may be used to translate and scale each airway based on thelung lobe change between respiration phases. The lung is constructed ofmultiple lobes and these lobes are commonly analyzed for volume, shape,and translation change. Each lobe changes in a very different way duringthe patient's respiration cycle. Using the respiratory-gated point cloudinformation to scale and translate the airways that are located in eachlobe, it is possible to adapt for airway movement. This scaled airwaymodel can then be linked to the 4D tracking of the patient as describedherein.

In general, it may also be preferable to reduce the level of radiationthat patients are exposed to before or during a procedure (orpre-procedural analysis) as described herein. One method of reducingradiation during the acquisition of a 3D fluoroscopic dataset (or otherdataset described herein), for example, is to use a deformation vectorfield between data points in a respiratory-gated point cloud to reducethe actual number of 2D images that need to be acquired to create the 3Ddataset. In one particular embodiment, the deformation field is used tocalculate the deformation between images in the acquisition sequence toproduce 2D images between the acquired slices, and these new slices canbe used to calculate the 3D fluoroscopic dataset. For example, if 180 2Dimage slices were previously required, e.g., an image(s) taken every 2degrees of a 360 degree acquisition sequence, in accordance with someembodiments 90 2D images can be acquired over a 360 degree acquisitionsequence and the data from the images that would have ordinarily beenacquired between each slice can be calculated and imported into the 3Dreconstruction algorithm. Thus, the radiation is effectively reduced by50%.

In another embodiment, illustrated by FIG. 10, a process of registrationand deformation may assist the navigation of a surgical instrument. Onemethod of preparing a segmented image dataset to match the anatomy of apatient's respiratory system comprises the steps of (i) forming (see box700) a respiratory-gated point cloud of data that demarcates anatomicalfeatures in a region of a patient's respiratory system at one or morediscrete phases within a respiration cycle of a patient, (ii) densityfiltering (see box 702) the respiratory-gated point cloud, (iii)classifying (see box 703) the density filtered respiratory-gated pointcloud according to anatomical points of reference in a segmented imagedataset for the region of the patient's respiratory system, (iii)registering (see box 800) the classified respiratory-gated point cloudto the segmented image dataset, (iv) comparing (see box 802) theregistered respiratory-gated point cloud to a segmented image dataset todetermine the weighting of points comprised by the classifiedrespiratory-gated point cloud, (v) distinguishing (see box 804) regionsof greater weighting from regions of lesser weighting and optionallyincreasing the data set comprised by the registered respiratory-gatedpoint cloud for regions of lesser weighting, and (vi) modifying ordeforming (see box 704) the segmented image dataset to correspond to theclassified respiratory-gated point cloud. In alternative embodiments,the user may optionally perform a loop 806 and generate additional datapoints in the respiratory-gated point cloud to increase the weighting ofcertain points in the respiratory-gated point cloud. In certainembodiments, the phases at which the respiratory-gated point cloud isformed include inspiration, expiration and phases in between.

In addition to modifying or deforming the segmented image dataset, inone embodiment of the present invention the movement of a patient'srespiratory system in the patient's respiration cycle over the patient'sentire respiration cycle may be simulated in a method comprising (i)forming (see box 700 of FIG. 10) a respiratory-gated point cloud of datathat demarcates anatomical features in a region of a patient'srespiratory system at one or more discrete phases within a respirationcycle of a patient, (ii) density filtering (see box 702 of FIG. 10) therespiratory-gated point cloud, (iii) classifying (see box 703 of FIG.10) the density filtered respiratory-gated point cloud according toanatomical points of reference in a segmented image dataset for theregion of the patient's respiratory system, (iv) creating a cine loopcomprising a plurality of modified segmented image datasets throughmultiple modifications of the segmented image dataset to correspond to aplurality of classified anatomical points of reference in therespiratory-gated point cloud over the respiration cycle, and (v)displaying the cine loop comprising the plurality of modified segmentedimage datasets over the patient's respiration cycle. In certainembodiments, the phases at which the respiratory-gated point cloud isformed include inspiration, expiration and phases in between. In certainembodiments, the plurality of modified segmented image datasets may becreated by modifying a segmented image dataset according to thedeformation vector field. In yet other embodiments, this simulatedmovement of the patient's respiratory system can be synchronized withthe patient's respiration cycle. Accordingly, the gating informationfrom the respiratory-gated point cloud is matched to a real-time gatingsignal corresponding to the patient's respiration cycle. The physiciancan then observe the modified or deformed image during the medicalprocedure on a targeted portion of the patient's body. Thus, during themedical procedure, the above simulation process can be continuouslyexecuted such that multiple modified images are displayed and modifiedimages corresponding to real-time positions of the patient's body can beviewed. In certain embodiments, the plurality of modified segmentedimage datasets comprises 2 or more segmented image datasets. In otherembodiments, the plurality of modified segmented image datasetscomprises 3 or more segmented image datasets. In yet other embodiments,the plurality of modified segmented image datasets comprises 4 or moresegmented image datasets. In certain embodiments, the creation anddisplay of the cine loop comprising the plurality of modified segmentedimage datasets can be over the patient's entire respiratory cycle.

In another aspect, the system involves generating a respiratory-gatedpoint cloud of a dynamic anatomy using implanted localization elements.In general, one or more (and typically multiple, e.g., 2, 3, 4, or more)localization elements may be placed in the organ and trackedcontinuously and registered to a discrete section of the organ. In thisembodiment, the localization elements may have a pigtail or anchoringmechanism that allows it to be attached to an internal organ or along avessel. Using image processing techniques, voxels from an image dataset,or set of voxels from an image dataset; multiple 3D data sets of theorgan can be used to create discrete sections of the organ (i.e., in agrid-like pattern). For each section, a deformation vector fieldanalysis can be performed between the phases of the organ and/or basedupon the motion of the organ tracked by localization elements attachedto or adjacent to a wall of the organ such that the motion of the organis translated to the sensors. Each section will then have uniquesignature or deformation vector field, which can be matched to thetracked motion of the localization element(s) attached to the organ. Forexample, the wall localization element motion will match the space-timesignature of the device. Preferably, a deformation vector field iscreated between at least two datasets. The deformation vector field maythen be applied to the segmented vessels and/or airways and the user'splanned path and target.

Another technique for maximizing registration accuracy is a centroidfinding algorithm that can be used for refining point locations in alocal area. Often, a user will want to select a vessel bifurcation. Thevessel bifurcation will be seen as a bright white location on the CT andUS images. An algorithm can be used to help the user select the centerlocation for these locations. Once a user selects a point on the image,the local algorithm can be employed to find similar white voxels thatare connected and, for that shape in the 3D space, refine the point tothe centroid or any other point (such as, for example, the most anterioror most posterior point).

Skeletonization of the segmented image dataset can help refine therespiratory-gated point cloud. It may be difficult to capture arespiratory-gated point cloud that would match a patient image datasetdue to the inability to physically touch the airway wall in manyorientations. Therefore the system can use the calculated centerlinesbetween the dataset and the respiratory-gated point cloud to refineaccuracy. Various methods of skeletonization are well known in the artand can be applied to the image datasets of certain embodiments of thepresent invention.

In an alternative embodiment, registering the classifiedrespiratory-gated point cloud to the segmented image dataset comprisesregistering the classified respiratory-gated point cloud representing atleast one branch of the patient's respiratory system to correspondinganatomical points of reference in the registered segmented image dataset representing the branch(es) of the patient's respiratory system. Incertain embodiments, the classified respiratory-gated point cloudsections corresponding to the trachea, the right main bronchus (RMB),and the left main bronchus (LMB) are registered to a plurality ofbranches of the patient's respiratory system, wherein the plurality ofbranches comprise the trachea, the right main bronchus (RMB), and theleft main bronchus (LMB). Rotational shifts may be found through lumendata collection. Matching the trachea, RMB, and LMB in patient space andimage space will provide rotational registration refinement. While thelung is commonly defined as one organ, in certain embodiments separateregistrations between the right and left lung or even different lobes ofthe lung can provide additional refinement. Carina touch points can beused to perform translational shifts to the registration between patientspace and image space.

In one embodiment a lung atlas can be used to develop patient specificairway trees, lung regions, lobes, lymph nodes, vessels, and otherstructures. These structures can be key to things such as correctlystaging lung cancer. Correctly identifying the spread of cancer to lymphnodes can determine the best course of patient treatment. Recording thesampled locations to determine a consistent staging methodology andcorrectly identifying the region of the lung is key. A lung atlas canalso be used to automatically select registration points within apatient such as the Main Carina or other branch points that can betouched by the user to register a dataset to the patient space. Using alung atlas with an airway tree segmented, a patient specific airway treecan be determined by deforming the lung atlas to the patient's datasetto produce a patient specific airway tree. This can be used for anavigation pathway map, initialization points for other image processingsteps, or to produce an error metric for multiple algorithms.Accordingly, in certain embodiments, a lung atlas can be modified ordeformed according to the respiratory-gated point cloud.

In another embodiment, 3D image datasets of an organ (e.g., the heart orlung(s)) are segmented to determine a center line of the pathway, suchthat a string of points, shape or diameter of the pathway can bedetermined. Patient image information can be matched to the localizationinformation in order to match 3D image space to actual patient space.Thus, an airway shape may be provided along with discrete segmentsproviding shape, orientation, and location information.

In yet another embodiment, 3D image datasets of an organ (e.g., theheart or lung(s)) are segmented to determine a wall, inner surface, oreffective inner surface of the pathway, such that a shape or diameter ofthe pathway can be determined. An effective inner surface may be therepresentation of an airway that can be tracked based upon theinstrumentation used to collect points. An instrument dragged through orpassed through and airway may be limited to its ability to track exactlyalong the surface of the airway and is generally a fixed distance fromthe wall (i.e., a 5 mm diameter airway may only be tracked in a 3 mmdiameter space as there is a 1 mm offset of the sensor from theinstrument or device it is inserted to the outer wall). Patient imageinformation may be matched to the localization information in order tomatch 3D image space to actual patient space. Thus, an airway shape isprovided along with discrete segments providing shape, orientation, andlocation information.

In one embodiment, the tracking of therapy delivery such as energy,material, device, or drug is described. Delivery of a therapy for COPD,asthma, lung cancer and other lung diseases needs tracking of thedelivery location and/or pattern. This can be done over treatmentsessions (i.e., Bronchial Thermoplasty) or have a dynamically changingdose or energy (RF, cryo, microwave, steam, radiation, or drugs) basedon location and trajectory of the delivery device. Using the trackinglocation and trajectory to modify the dose or energy real-time isdescribed. The power of an ablation device can be changed as the deviceis directed at the target or can be turned off if outside a definedregion. For delivery of therapy that is delivered over multiplesessions, the recorded locations of treated areas can be merged togetherfor each session to give the patient a complete treatment. Thetreatments can also be modeled before delivery to determine a moreeffective delivery pattern, dose, or energy.

In another embodiment, a catheter used in the forming of therespiratory-gated point cloud can be integrated with one or more fiberoptic localization (FDL) devices and/or techniques. In this way, thelocalization element (such as an electromagnetic (EM) sensor) providesthe 3D spatial orientation of the device, while the FDL provides shapesensing of the airway, vessel, pathway, organ, environment andsurroundings. Conventional FDL techniques can be employed. In variousembodiments, for example, the FDL device can be used to createlocalization information for the complete pathway or to refine thelocalization accuracy in a particular segment of the pathway. By eitherusing 3D localization information, shape, or both detected by the FDLdevice, the system can use a weighted algorithm between multiplelocalization devices to determine the location and orientation of theinstrument in the patient. The FDL device can also be used as or inconjunction with the PTD to track the patient's motion such asrespiration or heartbeat.

In other embodiments, surgical instrument 12 (see FIG. 1) may be abronchoscope that can capture a video view. This embodiment maycomprise, a guidewire or other navigated instrument with one to onerotation to continuously align a virtual display view to be consistentwith the actual bronchoscopic video view. A similar technique can beused with OCT, IVUS, or EBUS devices to orient the virtual view to theimage captured by the OCT, IVUS, or EBUS devices.

Still other embodiments involve using video input of the bronchoscope toadjust the virtual “fly-through” view to be consistent with the user'snormal perspective. For example, conventional video processing andmatching techniques can be used to align the real-time video and thevirtual image.

Still other embodiments involve using bronchoscopic video to provideangular information at a current location to provide targeting ordirectional cues to the user. Angular information can be derived fromthe location of patient anatomy in the image and the relative size ofeach within the image. Using information extracted from the videocaptured by the bronchoscope, the system can determine the direction ofthe display. This can be done using, for example, translation, rotation,or a combination of both. By comparing the real-time image captured tothe modified image constructed from the respiratory-gated point cloud,the system can use this information to align the modified image and/orenhance the system accuracy.

In yet another embodiment, a high-speed three-dimensional imagingdevice, such as an optical coherence tomography (OCT) device, can betracked. In accordance with conventional methods, such a device can onlyview 1-2 mm below the surface. With a localization element (e.g.,electromagnetic sensor) attached in accordance with the systems andmethods described herein, multiple 3D volumes of data can be collectedand a larger 3D volume of collected data can be constructed. Knowing the3D location and orientation of the multiple 3D volumes will allow theuser to view a more robust image of, for example, pre-cancerous changesin the esophagus or colon. This data can also be correlated topre-acquired or intra-procedurally acquired CT, fluoroscopic,ultrasound, or 3D fluoroscopic images to provide additional information.

Among several potential enhancements that could be provided by asurgical instrument navigation system as described herein is that a usercould overlay the planned pathway information on to the actual/real-timevideo image of the scope or imaging device (such as ultrasound baseddevice). Additionally, the system and apparatus could provide a visualcue on the real-time video image showing the correct direction orpathway to take.

According to another particular embodiment, 3D location information maybe used to extend the segmented airway model. The 3D airway can beextended as the instrument is passed along the airway by using thislocation information as an additional parameter to segment the airwayfrom the CT data. Using an iterative segmentation process, for instance,the 3D location information of the instrument can be used to provideseed points, manual extension, or an additional variable of likelihoodof a segmented vessel or airway existing in the 3D image volume. Theseadded airways can be displayed in a different format or color (forexample) or some other visual indicia to indicate to the user that theyare extending the segmented airway using instrument locationinformation.

The multi-dimensional imaging modalities described herein may also becoupled with digitally reconstructed radiography (DRR) techniques. Inaccordance with a fluoroscopic image acquisition, for example, radiationpasses through a physical media to create a projection image on aradiation-sensitive film or an electronic image intensifier. Given a 3Dor 4D dataset as described herein, for example, a simulated image can begenerated in conjunction with DRR methodologies. DRR is generally knownin the art, and is described, for example, by Lemieux et al. (Med. Phys.21(11), Nov. 1994, pp. 1749-60).

When a DRR image is created, a fluoroscopic image is formed bycomputationally projecting volume elements, or voxels, of the 3D or 4Ddataset onto one or more selected image planes. Using a 3D or 4D datasetof a given patient as described herein, for example, it is possible togenerate a DRR image that is similar in appearance to a correspondingpatient image. This similarity can be due, at least in part, to similarintrinsic imaging parameters (e.g., projective transformations,distortion corrections, etc.) and extrinsic imaging parameters (e.g.,orientation, view direction, etc.). The intrinsic imaging parameters canbe derived, for instance, from the calibration of the equipment.

Referring now to FIGS. 11A and 11B, in one embodiment of the presentinvention, the generated DRR image simulates bi-planar fluoroscopywherein an image of a region (or tissue) of interest 1602 near the ribs1208 of a patient 13 can be displayed for two planes of a 3D or 4D imagedataset. In certain embodiments, the two image planes of the simulatedbi-planar fluoroscopy are oriented 90 degrees apart, however it isunderstood that other angles are contemplated. One image view that maybe displayed is the anterior-to-posterior (A-P) plane (FIG. 11A) andanother image view that may be displayed is the lateral plane (FIG.11B). The display of these two image views provides another method tosee the up-and-down (and other directional) movement of the surgicalinstrument 12 (not shown). This simulated bi-planar fluoroscopy furtherprovides the ability to see how the surgical instrument 12 moves in animage(s), which translates to improved movement of surgical instrument12 in a patient 13. Additional information can be simultaneouslyintegrated with the simulated bi-planar fluoroscopy (in A-P and lateralviews) using minP (minimum intensity projection or maxP (maximumintensity projection) volume, including one or more of: (i) thesegmented airway 1601, (ii) the region (or tissue) of interest 1602,(iii) a real-time or simulated real-time rendering of the trajectory andlocation 1606 of surgical instrument 12, (iv) and the historical pathway1204 of the surgical instrument 12. Displaying the historical pathway1204 of surgical instrument 12 can assist the navigation of surgicalinstrument 12 in areas or situations in which there may be incompletesegmentation of the images of the patient's respiratory system. Inanother embodiment, a physician or other healthcare professional may beable to zoom in or out or pan through the simulated bi-planarfluoroscopy images.

According to another embodiment of the present invention, FIG. 12depicts a CT minP/maxP volume reading where precise navigation of asurgical instrument 12 (not shown) having a localization element 24(e.g., using an electromagnetic sensor) (not shown) near a region(s) (ortissue) of interest is carried out with incomplete segmentation results.This embodiment may provide the physician or other healthcareprofessional with a view or image 1600 using minP (minimum intensityprojection) or maxP (maximum intensity projection) volume renderings tosimultaneously integrate one or more of the segmented airway 1601, theregion(s) (or tissue) of interest 1602, and a visually distinctrepresentation of previously traversed paths that are “bad” (i.e.,incorrect) 1603. Additionally, this embodiment may also simultaneouslyintegrate the distance and angle 1604 to the region(s) (or tissue) ofinterest 1602 (e.g., a target lesion or tumor) using a vector fit to thelast 1 cm (or so) of travel (in addition to, or in place of,instantaneous orientation provided by a 5DOF localization element asdescribed herein) and may incorporate user provided “way points” tocreate a final-approach tube 1605 to the region(s) (or tissue) ofinterest 1602. As described herein, the image 1600 may also provide areal-time or simulated real-time rendering the trajectory and location1606 of surgical instrument 12 (e.g., the tip component as shown with avirtual extension 1607 to the region(s) (or tissue) of interest).

In another embodiment, an alternative method of generating a 4D datasetfor 4D thoracic registration using surgical instrument navigation system10 is illustrated by FIGS. 13A and 13B. In general, the 4D dataset maybe acquired through respiratory gating or tracheal temporalregistration. In accordance with the methods described herein, forexample, acceleration of N data collectors (e.g., magnetic or MEMSaccelerometers, for instance) are measured to register the thorax intime and space, using the general formula: dataT_(thorax)=F(t). As shownin FIGS. 13A and 13B, the various sensors 1701 and tracheal sensor 1702provide data as described herein, as does sternum sensor 1703 (e.g., x,y, and z dynamic tracking). The position and trajectory of a device(e.g., biopsy device or other device or medical instrument describedherein) is further capable of being tracked as described herein.

FIG. 14 shows another embodiment of an apparatus and method forrespiratory 4D data acquisition and navigation. As shown in the upperbox, a respiratory problem or issue is scanned (e.g., by a CT and/or MRscanner) and signal S from the tracking subsystem is provide to theCT/IR unit (lower box). The 4D registration based on the motion of thefiducial and tracker units (which could be, e.g., electromagneticsensors, MEMS devices, combinations thereof, and the like) is providedto the user (shown as an interventional radiologist (IR)) on computer C.The system is capable of displaying the current position of the devicetip in the image data shown to the IR, using the fiducial or trackerlocations in the scan coupled with the real-time motion informationprovided by the device tip (e.g., which can include a sensor asdescribed herein), thus providing registration.

Other embodiments include, for example, using an electromagnetic sensoras an LC or energy transmission device. This stored energy could be usedto actuate a sampling device such as forceps or power a diagnosticsensor.

In various aspects and embodiments described herein, one can use theknowledge of the path traveled by the surgical instrument and segmentedairway or vessel from the acquired image (e.g., CT) to limit thepossibilities of where the surgical instrument is located in thepatient. The techniques described herein, therefore, can be valuable toimprove virtual displays for users. Fly through, fly-above, or imagedisplays related to segmented paths are commonly dependent upon relativecloseness to the segmented path. For a breathing patient, for example,or a patient with a moving vessel related to heartbeat, the pathtraveled information can be used to determine where in the 4D patientmotion cycle the system is located within the patient. By comparing the3D location, the patient's tracked or physiological signal is used todetermine 4D patient motion cycle, and with the instrument's traveledpath, one can determine the optical location relative to a segmentedairway or vessel and use this information to provide the virtualdisplay.

The surgical instrument navigation system of certain embodiments of thepresent invention may also incorporate atlas maps. It is envisioned thatthree-dimensional or four-dimensional atlas maps may be registered withpatient specific scan data, respiratory-gated point clouds, or genericanatomical models. Atlas maps may contain kinematic information (e.g.,heart and lung models) that can be synchronized with four-dimensionalimage data, thereby supplementing the real-time information. Inaddition, the kinematic information may be combined with localizationinformation from several instruments to provide a completefour-dimensional model of organ motion. The atlas maps may also be usedto localize bones or soft tissue which can assist in determiningplacement and location of implants.

As noted herein, a variety of instruments and devices can be used inconjunction with the systems and methods described herein.

As a result of or in the course of certain surgical procedures, apatient's physical state may be changed relative to an acquired imagedataset. Incisions, insufflations, and deflation of the lung andre-positioning of the patient are just some of the procedures that maycause a change in the patient's physical state. Such changes in physicalstate may make it more difficult to find a lesion or point in an organof the patient. For example, in a lung wedge resection the thoracicsurgeon is palpating the lung to find the lesion to resect; if thislesion is 1-2 cm under the surface it can be very difficult to find.

In one embodiment, a first localization element is placed at a locationor region of interest (e.g., a tumor) within an organ of a patient and asecond localization element is used to identify the location of thefirst localization element from outside the organ in which the firstlocalization element has been positioned. Preferably, the firstlocalization element is attached or otherwise connected to tissue orsituated such that its position relative to the location or region ofinterest remains fixed. In some embodiments, for example, the firstlocalization element can be sutured in place, and/or or may have barbs,hooks, flexed spring shape (bowed) and/or wires, or other suitableconnection techniques, to hold it substantially in place.

Referring now to FIG. 15A, in one embodiment first localization element904 is attached to tissue in a region of the organ 900 of the patient.First localization element 904 may be placed, for example,percutaneously, endobronchially, or via the vasculature. As illustrated,first localization element may be wireless, however in other embodimentsfirst localization element may be wired. In one embodiment, firstlocalization element 904 is placed using an endolumenal device 902. Inpercutaneous (or other) methods, the leads of the first localizationelement may exit the patient and the device removed during or after aprocedure (e.g., during resection of a tumor).

In one embodiment, first localization element 904 is positioned in theorgan and may be registered to a segmented image dataset prior to anyprocedural resection or incision has occurred. Otherwise, pre-proceduralimages may not match the patient's anatomy (e.g., once an incision ismade, the patient is insufflated for a VATS procedure, or the patient isotherwise re-positioned).

After the first localization element 904 is positioned in an organ andregistered, as shown in FIG. 15B, the patient may then be manipulated ina manner that would potentially induce a physical change that wouldcause the patient's anatomy to not match a pre-procedure acquiredsegmented image dataset. A second localization element 908 (e.g., apointer probe) may then be used to identify the location of the firstlocalization element 904 from outside the patient's organ 900.Typically, second localization element 908 is placed near patient'sorgan 900 or other area of interest, e.g., via an incision or through aworking port of a VATS procedure or otherwise in a position to locatefirst localization element 904.

Although the second localization element 908 will be outside the organinto which first localization element 904 is placed, it need not beoutside the body of the patient. In certain embodiments of the presentinvention, second localization element 908 can be inserted into thepatient through a surgical portal. In other embodiments, secondlocalization element 908 will be outside the body of the patient.Optionally, and as illustrated in FIG. 15, in one embodiment a thirdlocalization element 906 may be used to locate first localizationelement 904 and second localization element 908 relative to thirdlocalization element 906, wherein the third localization element 906 maycomprise a 3D localizing device (e.g., an electromagnetic fieldgenerator) with a 3D coordinate system. In another embodiment, thirdlocalization element 906 may be used to locate second localizationelement 908 relative to third localization element 906.

As illustrated in FIG. 15, in one embodiment, the organ to which firstlocalization element 904 is attached is a lung. In other embodiments,however, first localization element 904 may be attached in another organsuch as a kidney or the liver.

In certain embodiments, the first, second and (optional) thirdlocalization elements may all be elements of a tracking subsystem 20(see FIG. 1). If tracking subsystem 20 is an electromagnetic trackingsystem, the third localization element would typically comprise anelectromagnetic field generator (transmitter) that emits a series ofelectromagnetic fields designed to engulf the patient, and the first andsecond localization elements could be coils that would receive(receivers) an induced voltage that could be monitored and translatedinto a coordinate position. However, the positioning of theelectromagnetic field generator (transmitter), and the first and secondlocalization elements (receivers) may also be reversed, such that thefirst localization element is a generator and the second and thirdlocalization elements are receivers or the second localization elementis a generator and the first and third localization elements arereceivers. Thus in certain embodiments, first localization element 904may be a receiver, while in other embodiments, first localizationelement 904 may be a receiver. In certain embodiments, secondlocalization element 908 may be a receiver, while in other embodiments,second localization element 908 may be a receiver. In certainembodiments, third localization element 906 may be a receiver, while inother embodiments, third localization element 906 may be a receiver.

In one embodiment of the present invention, surgical instrument 12 (seeFIG. 1) comprises a surgical catheter that is steerable (referred hereinto as “steerable catheter”) to gain access to, manipulate, remove orotherwise treat tissue within the body including, for example, heart orlung tissue. Generally, steerable catheters can be remotely manipulatedvia a steering actuator. In a typical medical procedure, the steeringactuator is located outside of the patient's body and is manipulated inorder to steer the steerable catheter to a desired location within thebody.

In accordance with one embodiment of the present invention and referringnow to FIG. 16, steerable catheter 200 comprises actuating handle 216and elongate flexible shaft 230. Elongate flexible shaft 230 hasproximal end portion 232, distal end portion 234, central longitudinalaxis 207 extending from proximal end portion 232 to distal end portion234, and outer wall 236 comprising a biocompatible material extendingfrom proximal end portion 232 to distal end portion 234. In certainembodiments, the biocompatible material is a biocompatible polymer.

In certain embodiments, elongate flexible shaft comprises a flexibleshaft portion 202 at its proximal end portion 232 and a steerable shaftportion 203 at its distal end portion 234. In other embodiments,elongate flexible shaft 230 comprises flexible shaft portion 202 at itsdistal end portion 234 and a steerable shaft portion at its proximal endportion 232. Flexible shaft portion 202 has a first stiffness andsteerable shaft portion 203 has a second stiffness that is less than thefirst stiffness. Stated differently, flexible shaft portion 202 may becomprised of a more rigid material which has a first stiffness, whilesteerable shaft portion 203 may be comprised of a softer material havinga second stiffness. In certain embodiments, flexible shaft portion 202is formed from a relatively high-durometer material and steerable shaftportion 203 is formed from a less stiff, lower-durometer material thanthe flexible shaft portion. Additionally, flexible shaft portion 202 maybe reinforced with a molded-in braided reinforcement material. In onealternative embodiment, flexible shaft portion 202 comprises a springhaving a first coil diameter and steerable shaft portion 203 comprises aspring having a second coil diameter. The first coil diameter may begreater than the second coil diameter and, accordingly, the first coildiameter of the flexible shaft portion has a greater stiffness than thesecond coil diameter of the steerable shaft portion. In one embodiment,elongate flexible shaft 230, including flexible shaft portion 202 andsteerable shaft portion 203, are preferably formed from a biocompatiblematerial such as Pebax™, manufactured by Arkema.

Biopsy device 220 is at distal end portion 234 of elongate flexibleshaft 230 and, in certain embodiments, may be used to access ormanipulate tissue. In one embodiment, biopsy device 220 is operated byactuation wire 212 (see FIG. 16A) which is housed within actuationchannel 209 extending through elongate flexible shaft 230. Actuationwire 212 has a proximal end (not shown) attached to handle 216 and adistal end (not shown) attached to biopsy device 220. As illustrated inFIG. 17A and as described in greater detail elsewhere herein, a varietyof biopsy devices 220 can be used with the steerable catheter,including, but not limited to, for example, a forceps device 17B, anauger device 17E, a boring bit device 17C, an aspiration needle device17F, or a brush device 17D. In one embodiment, biopsy device 220 iscomprised by a side exiting tip component 17G comprising a forcepsdevice, and auger device, a boring bit device, a brush device, or anaspiration needle device as described in greater detail elsewhereherein.

Referring again to FIG. 16, steerable catheter 200 further includes asteering mechanism comprising steering actuator 218 (proximate actuatinghandle 216) and at least one pull wire 210 (see FIG. 16A) housed inelongate flexible shaft 230 and attached to steering actuator 218. Incertain embodiments, manipulation of steering actuator 218 applies atension to pull wire(s) 210 and effects a deflection of steerable shaftportion 203 (located at or near the distal end portion of elongateflexible shaft 230) relative to flexible shaft portion 202. In certainembodiments, pull wire 210 extends the entire length of elongateflexible shaft 230. In other embodiments, pull wire 210 may extend onlyinto proximal end portion 232 of elongate flexible shaft 230. In yetother embodiments, pull wire 210 may extend only into distal end portion234 of elongate flexible shaft 230. In certain embodiments, pull wire210 is operably connected at its proximal end to steering actuator 218and anchored at its distal end to biopsy device 220 mounted on distalend portion 234 of elongate flexible shaft 230. Thus, pull wire 210passes through the flexible shaft portion and the steerable shaftportion of the elongate flexible shaft. The material for pull wire 210may be any suitable material usable with a catheter, such as stainlesssteel wire.

Referring now to FIG. 18A, distal end portion 234 may be deflectedrelative to proximal end portion 232 such that an arc β of at least 20degrees may be introduced into elongate flexible shaft 230 bymanipulation of steering actuator 218. As shown in FIGS. 18A and 18B,biopsy device is shown as an aspiration needle device 1100 (described ingreater detail elsewhere herein). In other embodiments, as described ingreater detail elsewhere herein, biopsy device 220 may comprise any of arange of other biopsy devices such as an auger device, a boring bitdevice, a brush device, a side exiting tip component, etc. In oneembodiment, an arc of at least about 30 degrees (i.e., β is at least 30degrees) may be introduced into elongate flexible shaft 230. By way offurther example, an arc of at least about 40 degrees may be introducedinto elongate flexible shaft 230. By way of further example, an arc ofat least about 45 degrees may be introduced into elongate flexible shaft230. By way of further example, an arc of at least about 60 degrees maybe introduced into elongate flexible shaft 230. By way of furtherexample, an arc of at least about 70 degrees may be introduced intoelongate flexible shaft 230. By way of further example, an arc of atleast about 80 degrees may be introduced into elongate flexible shaft230. By way of further example, an arc of at least about 90 degrees maybe introduced into elongate flexible shaft 230. By way of furtherexample, an arc of at least about 100 degrees may be introduced intoelongate flexible shaft 230. By way of further example, an arc of atleast about 110 degrees may be introduced into elongate flexible shaft230. By way of further example, an arc of at least about 120 degrees maybe introduced into elongate flexible shaft 230. By way of furtherexample, an arc of at least about 130 degrees may be introduced intoelongate flexible shaft 230. By way of further example, an arc of atleast about 140 degrees may be introduced into elongate flexible shaft230. By way of further example, an arc of at least about 150 degrees maybe introduced into elongate flexible shaft 230. By way of furtherexample, an arc of at least about 160 degrees may be introduced intoelongate flexible shaft 230. By way of further example, an arc of atleast about 170 degrees may be introduced into elongate flexible shaft230. By way of further example, an arc of about 180 degrees may beintroduced into elongate flexible shaft 230.

As illustrated in FIG. 18B, in one embodiment, manipulation of steeringactuator 218 introduces an arc β of at least 180 degrees into elongateflexible shaft 230. Distal end portion 234 may be moved such that adistance of no more than 1.5 inch separates two regions of elongateflexible shaft 230 located at opposing ends of a chord X connecting totwo points separated by at least 180 degrees on the arc. In certainembodiments, a distance of no more than 1 inch separates two regions ofelongate flexible shaft 230 located at opposing ends of chord X. Incertain embodiments, a distance of no more than 0.75 inches separatestwo regions of elongate flexible shaft 230 located at opposing ends ofchord X. In certain embodiments, a distance of no more than 0.5 inchesseparates two regions of elongate flexible shaft 230 located at opposingends of chord X. In certain embodiments, a distance of no more than 0.25inches separates two regions of elongate flexible shaft 230 located atopposing ends of chord X. In other embodiments, manipulation of steeringactuator 218 introduces an arc β of at least 120 degrees into elongateflexible shaft 230. Distal end portion 234 may be moved such that adistance of no more than 1.5 inch separates two regions of elongateflexible shaft 230 located at opposing ends of a chord X connecting totwo points separated by at least 120 degrees on the arc. In certainembodiments, a distance of no more than 1 inch separates two regions ofelongate flexible shaft 230 located at opposing ends of chord X. Incertain embodiments, a distance of no more than 0.75 inches separatestwo regions of elongate flexible shaft 230 located at opposing ends ofchord X. In certain embodiments, a distance of no more than 0.5 inchesseparates two regions of elongate flexible shaft 230 located at opposingends of chord X. In certain embodiments, a distance of no more than 0.25inches separates two regions of elongate flexible shaft 230 located atopposing ends of chord X.

In another embodiment, elongate flexible shaft 230 of steerable catheter200 houses more than one pull wire 210 attached to steering actuator218. The use of multiple pull wires may be preferred in some embodimentsover steerable catheters having a single pull wire. A steerable catheterhaving only one pull wire 210 attached to steering actuator 218 willtypically bend in only one direction, commonly referred to asuni-directional steering. A steerable catheter capable of onlyuni-directional steering could be rotated, such that any pointsurrounding the distal end of the elongate flexible shaft may be reachedby bending the catheter tip and rotating the catheter. Two or more pullwires (e.g., two, three, four, or even more) attached to steeringactuator 218, however, could provide multi-directional steering therebypermitting the elongate flexible shaft to be deflected in two or moredirections.

In one embodiment, elongate flexible shaft 230 comprises one or morelumens extending from proximal end portion 232 to distal end portion 234of elongate flexible shaft 234 that may be used to deliver a medicaldevice or therapy to a surgical site (e.g., fluids, biopsy devices,drugs, radioactive seeds, combinations thereof, or the like). In otherembodiments, the lumen(s) may house additional structures such aselectrical wires or optical fibers connected to biopsy device 220 ondistal end portion 234 of elongate flexible shaft 230. In otherembodiments, a vacuum pressure may be applied to the lumen(s) to assistremoval of tissue or fluid. In certain embodiments, the lumen may be aworking channel in which a biopsy device such as an aspiration needle ishoused and operated, wherein the aspiration needle is described ingreater detail elsewhere herein (see FIG. 33).

Referring now to FIG. 19, in another embodiment elongate flexible shaft230 of steerable catheter 200 comprises articulated spline 204containing a plurality of spline rings 206. Spline rings 206 are affixedin series to at least one hollow spline guide 208 and extend in thedirection of longitudinal central axis 207. Articulated spline 204 maybe covered in a casing 214 comprising a biocompatible material. In otherembodiments, articulated spline 204 is at distal end portion 234 ofelongate flexible shaft 230. In yet other embodiments, articulatedspline 204 is at proximal end portion 232 of elongate flexible shaft230. In yet other embodiments, steerable shaft portion 203 comprisesarticulated spline 204.

In one embodiment, as illustrated in FIG. 19, steerable catheter 200further comprises forceps device 300 as the biopsy device 220. In otherembodiments, as described in greater detail elsewhere herein, biopsydevice 220 may comprise any of a range of other biopsy devices such asan auger device, a boring bit device, a brush device, a side exiting tipcomponent, etc. Forceps device 300 comprises forceps housing 301, firstand second forceps jaws 302, localization element 24 and localizationelement lead wire 103, wherein the first and second forceps jaws 302 areoperably connected to actuation wire 212. The physician or otherhealthcare professional actuates forceps device 300 by manipulatinghandle 216 causing first and second forceps jaws 302 to pivot relativeto one another, thereby closing first and second forceps jaws 302thereby removing tissue. In certain embodiments, forceps device 300 mayalso comprise a tissue collection region where the removed tissue can becollected. In other embodiments, forceps device 300 can be equippedwith, or used in conjunction with, a vacuum pressure (suction) may beused to pull the removed tissue into tissue collection region. In yetother embodiments, tissue collection region of forceps device 300 mayhave a viewing window through which the removed tissue can be inspectedfrom outside forceps device 300 (see FIG. 27). In other embodiments,localization element 24 may be attached to actuation wire 212 such thatmovement of actuation wire 212 as handle 215 is manipulated causescoordinated movement of localization element 24 thereby providing anindication that forceps device 300 is being operated. A number ofmechanically operated forceps devices are known in the prior art and canbe adapted to operably connect to the actuation wire 212 of thesteerable catheter 200. As described in greater detail elsewhere herein,biopsy device 220 may comprise any of a range of other biopsy devicessuch as an auger device, a boring bit device, an aspiration needledevice, a brush device, a side exiting tip component, etc.

Referring now to FIG. 19A, hollow spline guides 208 are affixed toopposing inner surfaces of spline rings 206. Pull wires 210 are housedin hollow spline guides 208 with one pull wire housed within each hollowspline guide 208. The hollow spline guides 208 can be made of, forexample, stainless steel or other metal, or from a hard polymericmaterial, such as polyimide or PTFE, or from a polymer lined metal tube,such as a Teflon lined stainless steel tube. Each hollow spline guide208 may be made of a type of tube commonly used to fabricate hypodermicneedles, e.g., a stainless steel tube having an outside diameter ofabout 0.050 inches or less, and more preferably about 0.018 inches orless. This tubing is sometimes referred to as “hypotube.” By way ofexample, the guide tube may be a 26 gauge stainless steel hypodermictube, with a nominal outside diameter of 0.0183 inches and a nominalwall thickness of 0.004 inches. The hollow spline guides 208 provide andexhibit high strength and resiliency that resists compression. In oneembodiment, by way of example, hollow spline guides 208 are affixed tospline rings 206 by a laser weld. In another embodiment, by way offurther example, hollow spline guides 208 are affixed to spline rings206 by an epoxy. In another embodiment, by way of further example,hollow spline guides 208 are affixed to the spline rings 206 by solder.In another embodiment, by way of further example, hollow spline guides208 are affixed to spline rings 206 by a brazed joint.

Typically, the outer diameter of elongate flexible shaft 230 ofsteerable catheter 200 is less than 5 mm. By way of example, in certainembodiments, the outer diameter of elongate flexible shaft 230 ofsteerable catheter 200 is less than 1 mm. By way of further example, incertain embodiments, the outer diameter of elongate flexible shaft 230of steerable catheter 200 is less than 2 mm. By way of further example,in certain embodiments, the outer diameter of elongate flexible shaft230 of steerable catheter 200 is less than 3 mm. By way of furtherexample, in certain embodiments, the outer diameter of elongate flexibleshaft 230 of steerable catheter 200 is less than 4 mm. By way of furtherexample, in certain embodiments, the outer diameter of elongate flexibleshaft 230 of steerable catheter 200 is less than 5 mm.

While in certain embodiments the steerable catheter 200 isnon-navigated, other embodiments of the steerable catheter 200 arenavigated. In certain embodiments in which steerable catheter 200 isnavigated, a localization element 24 is positioned in elongate flexibleshaft 230 or biopsy device 220, preferably at or near the distal endthereof. In certain embodiments, localization element 24 may compriseelectromagnetic sensors. However, in other embodiments the steerablecatheter 200 may be navigated wherein elongate flexible shaft 230 orbiopsy device 220 may further comprise radiopaque markers visible viafluoroscopic imaging, or echogenic materials or patterns that increasevisibility of the tip component under an ultrasonic beam. In yet otherembodiments the steerable catheter 200 may be navigated wherein distalend portion 234 of elongate flexible shaft 230 or distal end of biopsydevice 220 may further comprise radiopaque markers visible viafluoroscopic imaging, or echogenic materials or patterns that increasevisibility of the tip component under an ultrasonic beam. In oneembodiment, localization element 24 comprises a six (6) degree offreedom (6DOF) electromagnetic sensor. In another embodiment thelocalization element comprises a five (5) degree of freedom (5DOF)electromagnetic sensor. Using the localization element, the user canhave the location of the biopsy device 220 is defined on the navigationscreen.

In one embodiment, localization element 24 may be attached to actuationwire 212. Movement of actuation wire 212 as handle 216 is manipulatedcauses coordinated movement of localization element 24 thereby providingan indication that biopsy device 220 is being operated. In accordancewith other embodiments, for example, a localization element as describedherein (e.g., an electromagnetic (EM) sensor) is affixed (preferablypermanently affixed, but may also be removable) to a biopsy device ormedical instrument so that both the biopsy device or medical instrument(or component thereof) and the localization element move together, suchthat they can be imaged and viewed. In one embodiment, for example,biopsy device 220 is an aspiration needle and the needle tip and thesensor move together. In another embodiment, for example, biopsy device220 is a brush, forceps, or forceps tissue capture mechanism and thesecomponents and the localization element move together. In these andother embodiments, handle 216 may be coupled with localization element24, thus allowing movement tracking. These various embodiments allow thebiopsy device or medical instrument (and components thereof) to betracked using the localization element, improving overall accuracy andreliability.

Referring now to FIGS. 20A, 20B and 20C, in one embodiment biopsy device220 further comprises an angled or directionally arranged radiopaquemarker pattern 115 that is visible via fluoroscopic imaging. Theradiopaque marker pattern 115 may be made of stainless steel, tantalum,platinum, gold, barium, bismuth, tungsten, iridium, or rhenium, alloysthereof, or of other radiopaque materials known in the art. Theradiopaque marker pattern 115 allows for tracking of the location andorientation of the biopsy device 220. Based upon the orientation of theradiopaque marker pattern 115 on the biopsy device 220 with respect tothe incident fluoroscopic beam, a distinct image is visible on afluoroscope. As the biopsy device 220 is navigated and rotated intoposition at the patient target, the orientation of the radiopaque markerpattern 115 with respect to the incident fluoroscopic beam will bealtered resulting in a corresponding change to the fluoroscopic image,thereby allowing the user to know the location and orientation of thebiopsy device 220. Accordingly, the user can then operate the biopsydevice 220 at the desired patient target. In other embodiments,radiopaque marker pattern 115 may be at distal end portion 234 ofelongate flexible shaft 230. In this embodiment, biopsy device 220 maycomprise any of the biopsy devices described elsewhere herein (see,e.g., FIGS. 17A-17G).

Referring now to FIGS. 21A, 21B, and 21C, in one embodiment biopsydevice 220 further comprises generally circular radiopaque markers 116placed around biopsy device 220, with 3 radiopaque markers on one sideof biopsy device 220 and 1 radiopaque marker on the opposite side ofbiopsy device 220. By way of example, in certain embodiments, 4radiopaque markers are placed on one side of biopsy device 220 and 2radiopaque markers are placed on the opposite side of biopsy device 220.By way of further example, in certain embodiments, 5 radiopaque markersare placed on one side of biopsy device 220 and 3 radiopaque markers areplaced on the opposite side of biopsy device 220. In other embodiments,circular radiopaque markers 116 may be at distal end portion 234 ofelongate flexible shaft 230. In this embodiment, biopsy device 220 maycomprise any of the biopsy devices described elsewhere herein (see,e.g., FIGS. 17A-17G).

Referring now to FIGS. 22 and 23, in one embodiment biopsy device 220further comprises an echogenic pattern that may be viewed via ultrasonicimaging. Several approaches of enhancing the ultrasonic signature ofmedical instruments through modification of the instrument surfacereflectivity are known in the prior art and can be applied toembodiments of the present invention. In one embodiment, an echogenicpattern can be positioned around the side wall of biopsy device 220,such that the echogenic pattern fully encompasses the exteriorcircumference of biopsy device 220. In other embodiments, an echogenicpattern may be at distal end portion 234 of elongate flexible shaft 230.In this embodiment, biopsy device 220 may comprise any of the biopsydevices described elsewhere herein (see, e.g., FIGS. 17A-17G).

Referring now to FIGS. 22 and 22A, the echogenic pattern comprises aplurality of partially spherical indentations 114 on the exteriorsurface of biopsy device 220, such that the radius of the partiallyspherical indentations is less than the wavelength of the incidentultrasonic beam. The plurality of partially spherical indentations 114cause constructive interference of the ultrasonic beam to affect anamplification of the reflecting beam along the line of the incidentbeam; such amplification may occur at any incident ultrasonic beamangle. In this embodiment, biopsy device 220 may comprise any of thebiopsy devices described elsewhere herein (see, e.g., FIGS. 17A-17G).

Referring now to FIGS. 23 and 23A, the echogenic pattern comprises aplurality of grooves 113 cut into the exterior surface of biopsy device220 that increase the reflective coefficient of biopsy device 220. Theplurality of grooves 113 may cause constructive interference of theultrasonic beam to affect an amplification of the reflecting beam alongthe line of the incident beam. In this embodiment, biopsy device 220 maycomprise any of the biopsy devices described elsewhere herein (see,e.g., FIGS. 17A-17G).

In certain embodiments, as discussed herein, localization element 24 maybe positioned at or near the distal end of biopsy device 220.Alternatively, in other embodiments, localization element 24 ispositioned at or near the proximal end of biopsy device 220. In yetother embodiments, multiple localization elements 24 (e.g., 5DOF or 6DOFelectromagnetic sensors) and/or radiopaque markers, echogenic patterns,etc. may be positioned at or near the proximal end of biopsy device 220.Alternatively, in other embodiments, multiple localization elements 24(e.g., 5DOF or 6DOF electromagnetic sensors) and/or radiopaque markers,echogenic patterns, etc. may be positioned at or near the distal end ofthe biopsy device 220. In yet other embodiments, multiple localizationelements 24 (e.g., 5DOF or 6DOF electromagnetic sensors) and/orradiopaque markers, echogenic patterns, etc. may be positioned at ornear the proximal and distal ends of the biopsy device 220. In anotherembodiment of the present invention, biopsy device 220 contains nolocalization element 24. Alternatively, localization element 24 ispositioned at or near distal end portion 234 of elongate flexible shaft230. By positioning localization element 24 at or near distal endportion 234 of elongate flexible shaft 230, in certain embodiments, thebiopsy device 220 could be made smaller or at a lesser cost. In thisembodiment, biopsy device 220 may comprise any of the biopsy devicesdescribed elsewhere herein (see, e.g., FIGS. 17A-17G).

In yet other embodiments, forceps device 300 (see FIG. 17B) may bevisible via fluoroscopic imaging wherein an angled or directionallyarranged radiopaque marker pattern 115 is at or near the proximal endand/or the distal end of forceps device 300 (see FIGS. 20A, 20B, and20C). In yet other embodiments, forceps device 300 may be visible viafluoroscopic imaging wherein a radiopaque marker pattern 115 comprisinggenerally circular radiopaque markers 116 is placed around forcepsdevice 300 (see FIGS. 21A, 21B, and 21C). In yet other embodiments,forceps device 300 may be visible via ultrasonic imaging wherein anechogenic pattern comprising a plurality of grooves 113 is in forcepsdevice 300 (see FIG. 23). In yet other embodiments, forceps device 300may be visible via ultrasonic imaging wherein an echogenic patterncomprising a plurality of partially spherical indentations 114 is inforceps device 300 (see FIG. 22).

Referring now to FIG. 24, in an alternative embodiment, steerablecatheter 200 comprises an alternative forceps device 300 as the biopsydevice 220. Instead of a mechanical forceps device actuated by anactuation wire (see FIG. 19), first and second forceps jaws 302 areoperated by a solenoid coil 1601. Solenoid coil 1601 is housed within inforceps housing 301 and when the solenoid is in an “active” mode, it isactivated by energy stored in capacitor 1603 thereby actuating the firstand second forceps jaws 302 via armature 1604. In a “passive” modesolenoid coil 1601 may act as a localization element 24 (e.g., anelectromagnetic or other sensor). In other embodiments, other biopsydevices as described elsewhere herein (see, e.g., FIGS. 17A-17G), may beactuated or operated in a similar manner.

Referring now to FIGS. 25A and 25B, in an alternative embodiment,steerable catheter 200 comprises an auger device 400 as the biopsydevice 220. Auger device 400 may be positioned proximate to the region(or tissue) of interest and is adapted to remove tissue from therespiratory system of a patient. In certain embodiments, auger device400 may comprise an auger bit housing 401, and an auger bit 402 withinthe auger bit housing 401. Auger bit 402 has a base and a tip, and atissue collection region 406 at the base of the auger bit 402, whereinthe auger bit 402 may be operably connected to the actuation wire 212.Auger bit 402 may be helical in shape. Auger bit housing 401 has aclosed proximal end attached to distal end portion 234 of elongateflexible shaft 230 and an open distal end. The proximal end of auger bithousing 401 has a hole through which the actuation wire extends towardthe open distal end. In certain embodiments, auger bit housing 401 andauger bit 402 can be made of, for example, stainless steel or othermetal, plastic, for example PVC, or from a hard polymeric material. Inone particular embodiment, the auger bit 402 has a constant diameterfrom the base to the tip. In another embodiment, the diameter of augerbit 402 decreases from the base to the tip. In this particularembodiment, as illustrated in FIG. 25B, auger bit 402 is extendable outthe open distal end of auger bit housing 401 and rotatable bymanipulation of the actuation wire 212 at handle 216 (not shown). Augerdevice 400 may be navigated via the inclusion of a localization element24 with a sensor lead 103 extending to proximal end portion 232 ofelongate flexible shaft 230. In other embodiments, localization element24 may be attached to actuation wire 212 such that movement of actuationwire 212 as handle 216 is manipulated causes coordinated movement oflocalization element 24 thereby providing an indication that augerdevice 400 is being operated. In other embodiments, localization element24 may be attached to auger bit 402. Once the target tissue of apatient's respiratory system is reached by steerable catheter 200, thephysician or other healthcare professional actuates auger device 400 bymanipulating handle 216 which actuates actuating wire 212 therebycausing auger bit 402 to extend and rotate, removing tissue from thepatient. Rotation of auger bit 402 draws the removed tissue into theauger bit housing 401 and into tissue collection region 406. In otherembodiments, a vacuum pressure may be used to pull the removed tissueinto tissue collection region 406. In yet other embodiments asillustrated by FIGS. 26A and 26B, auger device 400 may have nolocalization element.

As shown in FIG. 27, in another embodiment of auger device 400, tissuecollection region 406 of auger device 400 has a viewing window 408.Viewing window 408 allows for inspection, e.g., via bronchoscope orother viewing or sensing device such as described herein while thesteerable catheter 200 is still in the patient's body, of when thetissue collection region 406 is full or a sufficient sample size hasbeen collected. In other embodiments, the removed tissue can be viewedthrough viewing window 408 after the steerable catheter 200 has beenremoved from the patient's body or it can be viewed by a bronchoscopeinserted into the patient's body.

In yet other embodiments, auger device 400 may be visible viafluoroscopic imaging wherein an angled or directionally arrangedradiopaque marker pattern 115 is at or near the proximal end and/or thedistal end of auger device 400 (see FIGS. 20A, 20B, and 20C). In yetother embodiments, auger device 400 may be visible via fluoroscopicimaging wherein a radiopaque marker pattern 115 comprising generallycircular radiopaque markers 116 is placed around auger device 400 (seeFIGS. 21A, 21B, and 21C). In yet other embodiments, auger device 400 maybe visible via ultrasonic imaging wherein an echogenic patterncomprising a plurality of grooves 113 is in auger device 400 (see FIG.23). In yet other embodiments, auger device 400 may be visible viaultrasonic imaging wherein an echogenic pattern comprising a pluralityof partially spherical indentations 114 is in auger device 400 (see FIG.22).

Referring now to FIGS. 28A and 28B, in an alternative embodiment,steerable catheter 200 comprises a boring bit device 500 as the biopsydevice 220. Boring bit device 500 may be positioned proximate to theregion (or tissue) of interest and is adapted to remove tissue from therespiratory system of a patient. Additionally, in certain embodiments,elongate flexible shaft 200 further comprises a vacuum channel 506. Incertain embodiments, boring bit device 500 may comprise a boring bithousing 501 and a boring bit 502 within boring bit housing 501. Boringbit 502 may comprise a hollow cylinder having a closed proximal end andan open distal end having a plurality of cutting teeth around thecircumference of the cylinder, wherein boring bit 502 may be operablyconnected to actuation wire 212. Boring bit housing 501 and boring bit502 can be made of, for example, stainless steel or other metal,plastic, for example PVC, or from a hard polymeric material. Boring bitdevice 500 may be navigated via the inclusion of an attachedlocalization element 24 with a sensor lead 103 extending to the proximalend portion 232 of elongate flexible shaft 230. In other embodiments,localization element 24 may be attached to actuation wire 212 such thatmovement of actuation wire 212 as handle 216 is manipulated causescoordinated movement of localization element 24 thereby providing anindication that boring bit device 500 is being operated. In otherembodiments, localization element 24 may be attached to boring bit 502.In this particular embodiment, as illustrated in FIG. 28B, boring bit502 is extendable out the open distal end of boring bit housing 501 androtatable by manipulation of the actuation wire 212 at handle 216 (notshown). Once the target tissue of a patient's respiratory system isreached by steerable catheter 200, the physician or other healthcareprofessional actuates boring bit device 500 by manipulating handle 216which actuates actuating wire 212 thereby causing boring bit 502 toextend and rotate, removing tissue from the patient. In certainembodiments, boring bit device 500 may also comprise a tissue collectionregion where the removed tissue can be collected. In yet otherembodiments, tissue collection region of boring bit device 500 may havea viewing window through which the removed tissue can be inspected fromoutside boring bit device 500 (see FIG. 27). In certain embodiments,boring bit device 500 may have no localization element as illustrated inFIGS. 30A and 30B.

FIG. 29 illustrates another embodiment where, boring bit housing 501 mayhave at least one opening 510 in the proximal end attached to distal endportion 234 of elongate flexible shaft 230 and an open distal endthrough which the boring bit 502 is extended. Additionally, in thisembodiment, the proximal end of boring bit 502 may have at least oneopening 510. The proximal end of boring bit housing 501 has a holethrough which the actuation wire 212 extends toward the open distal end.In this particular embodiment, boring bit 502 is extendable out the opendistal end of the boring bit housing 501 and rotatable by manipulationof the actuation wire 212 at handle 216 (not shown). Once the targettissue of a patient's respiratory system is reached by the steerablecatheter 200, the physician or other healthcare professional actuatesboring bit device 500 by manipulating handle 216 which actuatesactuating wire 212 thereby causing boring bit 502 to extend and rotate,removing tissue from the patient. Additionally, a vacuum pressure may beapplied at proximal end portion 232 of elongate flexible shaft 230wherein this pressure acts on the vacuum channel 506, the at least oneopening 510 in the proximal end of boring bit housing 501, and the atleast one opening 510 in the proximal end of boring bit 502 to aid inthe removal of the patient's tissue. In other embodiments, whereinboring bit device 500 may further comprise a tissue collection region,the applied vacuum pressure may be used to pull the removed tissue intotissue collection region.

In another embodiment, as illustrated by FIGS. 31A and 31B, boring bithousing 501 has closed proximate end attached to the distal end portion234 of elongate flexible shaft 230 and an open distal end through whichboring bit 502 is extended. The proximal end of boring bit housing 501has a hole through which the actuation wire 504 extends toward the opendistal end. Additionally, in this embodiment, actuation wire 504 ishollow. In this particular embodiment, as illustrated in FIG. 32, boringbit 502 is extendable out the open distal end of boring bit housing 501and rotatable by manipulation of hollow actuation wire 504 at handle216. Once the target tissue of a patient's respiratory system is reachedby steerable catheter 200, the physician or other healthcareprofessional actuates the boring bit device 500 by manipulating handle216 which actuates hollow actuating wire 504 thereby causing boring bit502 to extend and rotate, removing tissue from the patient.Additionally, a vacuum pressure may be applied at proximal end portion232 of elongate flexible shaft 230 wherein this pressure acts on hollowactuation wire 504 to aid in the removal of the patient's tissue.

In yet other embodiments, boring bit device 500 may be visible viafluoroscopic imaging wherein an angled or directionally arrangedradiopaque marker pattern 115 is at or near the proximal end and/or thedistal end of boring bit device 500 (see FIGS. 20A, 20B, and 20C). Inyet other embodiments, boring bit device 500 may be visible viafluoroscopic imaging a radiopaque marker pattern 115 comprisinggenerally circular radiopaque markers 116 is placed around boring bitdevice 500 (see FIGS. 21A, 21B, and 21C). In yet other embodiments,boring bit device 500 may be visible via ultrasonic imaging wherein anechogenic pattern comprising a plurality of grooves 113 is in boring bitdevice 500 (see FIG. 23). In yet other embodiments, boring bit device500 may be visible via ultrasonic imaging wherein an echogenic patterncomprising a plurality of partially spherical indentations 114 is inboring bit device 500 (see FIG. 22).

Referring now to FIG. 33, in an alternative embodiment, steerablecatheter 200 comprises an aspiration needle device 1100 as the biopsydevice 220. Aspiration needle device 1100 may be positioned proximate tothe region (or tissue) of interest and is adapted to remove tissue fromthe respiratory system of a patient. In certain embodiments, aspirationneedle device 1100 may comprise an aspiration needle housing 1101 and anaspiration needle 1102 within aspiration needle housing 1101. Aspirationneedle device 1100 may be navigated via the inclusion of an attachedlocalization element 24 with a sensor lead 103 extending to proximal endportion 232 of elongate flexible shaft 230. In other embodiments,localization element 24 may be attached to hollow actuation wire 504such that movement of hollow actuation wire 504 as handle 216 ismanipulated causes coordinated movement of localization element 24thereby providing an indication that aspiration needle device 1100 isbeing operated. In other embodiments, localization element 24 may beattached to aspiration needle 1102. In certain embodiments, aspirationneedle device 1100 may have no localization element. In certainembodiments, aspiration needle 1102 may be between 18 and 22 ga.Aspiration needle housing 1101 and aspiration needle 1102 can be madeof, for example, nitinol, stainless steel or other metal, plastic, forexample PVC, or from a hard polymeric material. In certain embodiments,aspiration needle 1102 may be a flexible needle. In other embodiments,aspiration needle 1102 may be a relatively rigid (non-flexible) needle.In yet other embodiments, aspiration needle 1102 may comprise a shapememory alloy, as described in greater detail elsewhere herein. There area number of mechanically operated aspiration needle devices that areknown in the prior art and can be adapted to operably connect to ahollow actuation wire of the steerable catheter 200. In additionalembodiments, the aspiration needle device comprises a single biopsydevice that is attached at the proximal end portion 232 of elongateflexible shaft 230 and extends to the distal end portion 234 of elongateflexible shaft 232. Aspiration needle device 1100 may be navigated viathe inclusion of an attached localization element 24 with a sensor lead103 extending to proximal end portion 232 of elongate flexible shaft230. Once the target tissue of a patient's respiratory system is reachedby steerable catheter 200, the physician or other healthcareprofessional actuates aspiration needle device 1100 by manipulatinghandle 216 which actuates the actuating wire 504 thereby causingaspiration needle 1102 to extend and pierce tissue in the patient.Additionally, a vacuum pressure may be applied at proximal end portion232 of elongate flexible shaft 230 wherein this pressure acts on hollowactuation wire 504 to aid in the removal of the patient's tissue. Inanother embodiments, aspiration needle device 1100 comprises a singlebiopsy device that is attached at proximal end portion 232 of elongateflexible shaft 230 and extends to distal end portion 234 of elongateflexible shaft 230, the physician or other healthcare professionalactuates aspiration needle device 1100 by manipulating handle 216 whichcauses aspiration needle 502 to extend and pierce tissue from thepatient. Additionally, a vacuum pressure may be applied at proximal endportion 232 of elongate flexible shaft 230 wherein this pressure acts onaspiration needle 1102 to aid in the removal of the patient's tissue. Incertain embodiments, aspiration needle device 1100 may also comprise atissue collection region where the removed tissue can be collected. Inother embodiments, the applied vacuum pressure may be used to pull theremoved tissue into tissue collection region. In yet other embodiments,tissue collection region of aspiration needle device 1100 may have aviewing window through which the removed tissue can be inspected fromoutside aspiration needle device 1100 (see FIG. 27).

In yet other embodiments, aspiration needle device 1100 may be visiblevia fluoroscopic imaging wherein an angled or directionally arrangedradiopaque marker pattern 115 is at or near the proximal end and/or thedistal end of aspiration needle device 1100 (see FIGS. 20A, 20B, and20C). In yet other embodiments, aspiration needle device 1100 may bevisible via fluoroscopic imaging wherein a radiopaque marker pattern 115comprising generally circular radiopaque markers 116 is placed aroundaspiration needle device 1100 (see FIGS. 21A, 21B, and 21C). In yetother embodiments, aspiration needle device 1100 may be visible viaultrasonic imaging wherein an echogenic pattern comprising a pluralityof grooves 113 is in aspiration needle device 1100 (see FIG. 23). In yetother embodiments, aspiration needle device 1100 may be visible viaultrasonic imaging wherein an echogenic pattern comprising a pluralityof partially spherical indentations 114 is in aspiration needle device1100 (see FIG. 22).

Referring now to FIGS. 34A and 34B, in an alternative embodiment,steerable catheter 200 comprises a brush device 1000 as the biopsydevice 220. Brush device 1000 may be positioned proximate to the region(or tissue) of interest and is adapted to remove tissue from therespiratory system of a patient. In certain embodiments, brush device1000 may comprise a brush housing 1001 and a brush 1002 wherein brush1002 comprises a plurality of bristles affixed to an actuation wire 212.Brush housing 1001 may have a closed proximal end attached to the distalend portion 234 of elongate flexible shaft 230 and an open distal end.Additionally brush housing 1001 may have an internal wall 1003 with anaperture 1004 having a diameter less than that of the diameter of brush1002. The proximal end of brush housing 1001 has a hole through whichthe actuation wire 212 extends toward the open distal end. Brush housing1001 can be made of, for example, stainless steel or other metal,plastic, for example PVC, or from a hard polymeric material. Brush 1002is extendable out the open distal end of brush housing 1001 androtatable by manipulation of the actuation wire 212 at handle 216. Brushdevice 1000 may be navigated via the inclusion of an attachedlocalization element 24 with a sensor lead 103 extending to proximal endportion 232 of elongate flexible shaft 230. In other embodiments,localization element 24 may be attached to actuation wire 212 such thatmovement of actuation wire 212 as handle 216 is manipulated causescoordinated movement of localization element 24 thereby providing anindication that brush device 1000 is being operated. In otherembodiments, localization element 24 may be attached to brush 1002. Incertain embodiments, brush device 1000 may have no localization element.Once the target tissue of a patient's respiratory system is reached bysteerable catheter 200, the physician or other healthcare professionalactuates brush device 1000 by manipulating handle 216 which actuatesactuating wire 212 thereby causing brush 1002 to extend and rotate,removing tissue 1005 from the patient. The physician or other healthcareprofessional may then withdraw brush 1002 into brush housing 1001whereby removed tissue 1006 is scraped from brush 1002 by aperture 1004.Additionally, in another embodiment, elongate flexible shaft 230 furthercomprises a vacuum channel and the proximal end of brush housing 1001further comprises at least one hole to which a vacuum pressure can beapplied. Additionally, a vacuum pressure may be applied at proximal endportion 232 of elongate flexible shaft 230 wherein this pressure acts onthe vacuum channel and the at least one opening in the proximal end ofbrush housing 1001 to aid in the removal of the patient's tissue. Incertain embodiments, brush device 1000 may also comprise a tissuecollection region where the removed tissue can be collected. In otherembodiments, the applied vacuum pressure may be used to pull the removedtissue into tissue collection region. In yet other embodiments, tissuecollection region of brush device 1000 may have a viewing window throughwhich the removed tissue can be inspected from outside brush device 1000(see FIG. 27).

In yet other embodiments, brush device 1000 may be visible viafluoroscopic imaging wherein an angled or directionally arrangedradiopaque marker pattern 115 is at or near the proximal end and/or thedistal end of brush device 1000 (see FIGS. 20A, 20B, and 20C). In yetother embodiments, brush device 1000 may be visible via fluoroscopicimaging wherein a radiopaque marker pattern 115 comprising generallycircular radiopaque markers 116 is placed around brush device 1000 (seeFIGS. 21A, 21B, and 21C). In yet other embodiments, brush device 1000may be visible via ultrasonic imaging an echogenic pattern comprising aplurality of grooves 113 is in brush device 1000 (see FIG. 23). In yetother embodiments, brush device 1000 may be visible via ultrasonicimaging wherein an echogenic pattern comprising a plurality of partiallyspherical indentations 114 is in brush device 1000 (see FIG. 22).

In another embodiment of a brush device, as brush is pushed out of thebrush housing the brush is squeezed through a smaller opening to collectthe sampled tissue that was trapped when extended. When fully extended,the brush device end would be open so that the brush can be retractedand the sampled tissue can be pulled into the instrument. The brushdevice would then close as the brush is extended out and the sampledtissue could be scraped/squeezed from the brush bristles and collectedin a reservoir. In certain embodiments, a vacuum pressure may be addedto this device in conjunction or in lieu of the scrapping process toclean the brush.

In yet another embodiment, biopsy device 220 is extendable is extendablealong a path from a position within the outer wall 236 through a sideexit to a position outside the outer wall 236 at an angle of at least 30degrees relative to the longitudinal axis, wherein the path of biopsydevice 220 can be calibrated to the location of an electromagneticlocalization sensor positioned at the distal end portion of the elongateflexible shaft and displayed by a surgical instrument navigation system.Various embodiments of biopsy devices exiting from the side of theelongate catheter body can be seen in FIG. 37.

In these and other embodiments, a portion of biopsy device 220 may bebent at an angle relative longitudinal axis 207. A bend in biopsy device220, may allow the physician or other healthcare professional to rotatebiopsy device 220 and sample the region (or tissue) of interest viaseveral different pathways or positions. This may also increase theamount of region (or tissue) of interest that may be sampled in a singlepass, and may improve targeting of regions (or tissue) of interest to besampled. Moreover, a bend in biopsy device 220 may assist in targeting aregion that is not necessarily directly in a patient pathway (e.g., anairway), but may be next to the pathway, thus enabling the physician orother healthcare professional to direct biopsy device 220 to a desiredlocation off the axis of the airway.

The method or procedure of guiding the steerable catheter 200 of certainembodiments to a desired target tissue in the respiratory system of apatient comprises: (i) inserting a flexible lumen into the patient, (ii)inserting into the flexible lumen steerable catheter 200, (ii)navigating steerable catheter 200 through the respiratory system of thepatient, (iii) manipulating steering actuator 218 to cause a deflectionin longitudinal axis 207, and (iv) performing a medical procedure at theregion (or tissue) of interest. In embodiments where steerable catheter200 is a navigated catheter, the method or procedure of guiding thesteerable catheter 200 of certain embodiments to a desired target tissueof a patient includes the additional steps of: (i) displaying an imageof the region of the patient, (ii) detecting a location and orientationof localization element 24, and (iii) displaying, in real-time, biopsydevice 220 on the image by superimposing a virtual representation ofsteerable catheter 200 and biopsy device 220 on the image based upon thelocation and orientation of localization element.

A method or procedure of guiding steerable catheter 200 of certainembodiments to a desired target tissue in the respiratory system of apatient may comprise inserting a flexible lumen into the patient, andinserting into the flexible lumen steerable catheter 200. Steerablecatheter 200 may then be navigated to the region of interest andsteering actuator 218 may be manipulated to cause a deflection inlongitudinal axis 207. Medical procedure may then be performed at theregion (or tissue) of interest. In embodiments where steerable catheter200 is a navigated catheter, the method or procedure of guidingsteerable catheter 200 of certain embodiments to a region (or tissue) ofinterest may comprise the additional steps of displaying an image of theregion (or tissue) of interest and detecting a location and orientationof localization element 24. Then biopsy device 220 may be displayed, inreal-time, on the image by superimposing a virtual representation ofsteerable catheter 200 and biopsy device 220 on the image based upon thelocation and orientation of localization element 24.

In one embodiment of the present invention, surgical instrument 12 (seeFIG. 1) comprises a surgical catheter having a side exit (referred toherein as “side exiting catheter”) which may be used in a medicalprocedure to gain access to, manipulate, remove or otherwise treattissue within the body including, for example, heart or lung tissue.Generally, surgical catheters of the present invention have a distal endportion which can be remotely manipulated via a proximally locatedhandle.

In accordance with one embodiment of the present invention and referringnow to FIG. 35, side exiting catheter 100 comprises actuating handle 110and elongate flexible shaft 130. Elongate flexible shaft 130 hasproximal end portion 132, distal end portion 134, longitudinal axis 109extending from proximal end portion 132 to distal end portion 134, outerwall 136 comprising a biocompatible material extending from proximal endportion 132 to distal end portion 134, and a side exit 105 at distal endportion 134 of elongate flexible shaft. In certain embodiments, thebiocompatible material is a biocompatible polymer. The stiffnessproperties of elongate flexible shaft 130 allow elongate flexible shaft130 to be advanced in patient 13 in a desired direction to a desiredregion (or tissue of interest) via the application of torsional orlinear force applied to handle 110 at proximal end portion 132 ofelongate flexible shaft 134.

A localization element 24 may be positioned at distal end portion 134 ofelongate flexible shaft 130. In general any of a number of localizationelements 24 may be used, including, but not limited to, for example,electromagnetic sensors, radiopaque markers visible via fluoroscopicimaging, or echogenic materials or patterns that increase visibility ofthe tip component under an ultrasonic beam. In this embodiment,localization element 24 comprises a six (6) degree of freedom (6DOF)electromagnetic sensor. In other embodiments, localization element 24comprises a five (5) degree of freedom (5DOF) electromagnetic sensor. Alocalization element lead 103 extends from localization element 24 toproximal end portion 132 of elongate flexible shaft 130. In analternative embodiment, localization element 24 and the electromagneticfield generator may be reversed, such that localization element 24positioned at distal end portion 134 of elongate flexible shaft 130emits a magnetic field that is sensed by external sensors.

Side exiting catheter 100 further comprises medical instrument 108 thatmay be extended from a position within outer wall 136 and through sideexit 105 to a position outside outer wall 136 of elongate flexible shaft130 by manipulation of handle 110. For ease of illustration, only theportion of medical instrument 108 that is extended outside elongateflexible shaft 130 appears in FIG. 35; the remaining portion of medicalinstrument 108 which is attached to handle 110 is hidden from view (seeFIGS. 51A and 51B). In one embodiment, medical instrument 108 extendsthrough side exit 105 along a path at an angle of at least 10 degreesrelative to longitudinal axis 109.

In certain embodiments, by using localization element 24 (which incertain embodiments comprises a 6DOF sensor as described herein), thephysician or other healthcare professional can have the location anddirection of side exit 105 of elongate flexible shaft 130 displayed bysurgical instrument navigation system 10. A real time two- orthree-dimensional virtual reconstruction of side exit 105 and severalcentimeters of distal end portion 132 of elongate flexible shaft 130 maybe displayed by surgical instrument navigation system 10. Visualizationof the location and orientation of side exit 105 may allow for moreintuitive advancement of side exiting catheter 100 to a region (ortissue) of interest. An image plane may be generated that is at sideexit 105, as opposed to a point or position distal to distal end portion134 of elongate flexible shaft 130. In certain embodiments, this mayallow easier targeting of lesions, or other region(s) (or tissue) ofinterest, that may not be directly in the airway or other pathways, butrather partially or even completely outside of the airway or otherpathways. In accordance with an exemplary method of using the device,side exiting catheter 100 may be steered slightly past the region (ortissue) of interest to align side exit 105. Medical instrument 108(e.g., forceps, needle, brush, fiducial delivery device, etc.) may thenbe extended out elongate flexible shaft 130 through side exit 105. Thedirectional aspect of distal end portion 134 and medical instrument 108can be viewed on display 18 (see FIG. 1) and a simulated medicalinstrument can be shown to demonstrate to the physician or otherhealthcare professional which region (or tissue) of interest will besampled. These applications may be particularly useful in the samplingof lymph nodes that are outside the patient airways. In someembodiments, for example, side exiting catheter 100 may be capable ofcreating an endobronchial ultrasound (EBUS)-like view. For example, animage plane oriented with side exit 105 plane may be created and medicalinstrument 108 may be shown sampling the region (or tissue) of intereston this plane. In various alternative embodiments, the image(s) may beoriented in a plane or orthogonally. Additionally in other embodiments,various curve fitting algorithms may be provided based upon the type andflexibility of the elongate flexible shaft used. These algorithms enableestimated curved trajectories of elongate flexible shaft 130 to bedisplayed to assist the physician or other healthcare professional.

In accordance with another embodiment, systems and methods may be usedto provide the initial location of a localization element (e.g., anelectromagnetic sensor) in a surgical instrument (e.g., a steerablesurgical catheter, a side exiting catheter, a steering or shape sensingdevice such as a robotic articulating arm, fiber optic shape trackingdevice, or micro-actuator/flex systems, etc.). In one embodiment, acalibration jig system may be employed. The calibration jig systemcomprises at least three reference localization elements (e.g.,electromagnetic sensors) positioned substantially in a plane and atool/calibration channel may be positioned in a known location relativeto the localization element plane. A surgical instrument may then beinserted into the tool/calibration channel and the surgical instrumentpathway shape may be recorded along with the localization element(s)(e.g., 5DOF and/or 6DOF electromagnetic sensors) in the surgicalinstrument with respect to the location of the reference localizationelements in the localization element plane. By using this calibrationjig system, the alignment of the localization element(s) within thesurgical instrument may be determined relative to the alignment of thesurgical instrument. Additionally, or alternatively, other sensingmechanisms that report position and/or shape can be correlated relativeto the reference localization element coordinates and therefore maydefine the complete or substantially complete physical coordinatesystem. Because the position of the jig tool channel is known relativeto the position of the reference localization elements, the position ofsensing mechanisms placed within the surgical instrument may bedetermined relative to the calibration jig and therefore relative toeach other.

In certain embodiments, as illustrated in FIG. 36, after calibration ofthe alignment of localization element 24 (e.g., an electromagneticsensor) in side exiting catheter 100 using the calibration jig system,path 122 of a medical instrument (not shown) may be calibrated relativeto the position of localization element 24. During calibration, thelocation/coordinates of the tip of medical instrument 108 will bemeasured relative to the position of localization element 24 for aninitial position X₀. After medical instrument 108 is advanced inelongate flexible shaft 130 a distance X₁ to extend medical instrument108 through side exit 105 to a position outside the elongate flexibleshaft 130 the location/coordinates of the tip of medical instrument 108can be measured at x_(tip1), y_(tip1), and z_(tip1) relative to localcoordinate system 120 of localization element 24. Similarly, aftermedical instrument 108 is advanced in elongate flexible shaft 130 adistance X₂ to extend medical instrument 108 to a further positionoutside the elongate flexible shaft 130 the location/coordinates of thetip of medical instrument 108 can be measured at x_(tip2), y_(tip2),z_(tip2) relative to local coordinate system 120 of localization element24. As medical instrument 108 extends through side exit 105 (see FIG.35) the path 122 of medical instrument 108 is at angle Θ relative tolongitudinal axis 109. As such, the location/coordinates of the tip ofmedical instrument 108 can be measured at a plurality of positions(e.g., 2, 3, 4, or more) within and outside elongate flexible shaft 130.The calibration can be along the range of distances for which medicalinstrument 108 may be advanced. Accordingly, after path 122 of medicalinstrument 108 is determined, the path 122 can be displayed by surgicalinstrument navigation system 10 (see FIG. 1) as a virtual path to aidthe physician or other healthcare professional in extending the medicalinstrument to the desired region (or tissue) of interest.

Referring again to FIG. 35, in one embodiment, medical instrument 108can be a flexible instrument, which in certain embodiments, may be anaspiration needle. In certain embodiments, the flexible instrument maybe comprised of flexible nitinol. In another embodiment, medicalinstrument 108 can be constructed of a flexible coil having a stainlesssteel tip. In yet another embodiment, medical instrument 108 may be arelatively rigid (non-flexible). In yet other embodiments, medicalinstrument 108 may comprise a shape memory alloy, as described ingreater detail elsewhere herein. As illustrated in FIG. 37, medicalinstrument 108 may comprise a variety of biopsy devices at the end 140of medical instrument 108, including, but not limited to, for example, aforceps device 37A, an auger device 37D, a boring bit device 37B, and abrush device 37C. In addition to biopsy devices, in yet otherembodiments, a variety of medical instruments may be used such asfiducial delivery devices, diagnostic imaging devices (such as OCT), ortherapy devices (such as an ablation device, an RF emitter, a microwaveemitter, a laser device, a device to deliver a radioactive seed, acryogenic therapy delivery device, a drug delivery device, or a fluiddelivery device, among others), or combinations thereof, and the like.In other embodiments, medical instrument 108 may be used to delivertherapy to a surgical site (e.g., fluids, drugs, radioactive seeds,combinations thereof, or the like).

In one embodiment, as illustrated by FIG. 38, medical instrument 108extends through side exit 105 along a path at an angle Θ of at least 10degrees relative to longitudinal axis 109. Typically, medical instrument108 extends through side exit 105 along a path at an angle from about 10degrees to about 70 degrees relative to longitudinal axis 109. By way ofexample, in certain embodiments, medical instrument 108 extends throughside exit 105 along a path at an angle of about 10 degrees relative tolongitudinal axis 109. By way of further example, in certainembodiments, medical instrument 108 extends through side exit 105 alonga path at an angle of about 15 degrees relative to longitudinal axis109. By way of further example, in certain embodiments, medicalinstrument 108 extends through side exit 105 along a path at an angle ofabout 20 degrees relative to longitudinal axis 109. By way of furtherexample, in certain embodiments, medical instrument 108 extends throughside exit 105 along a path at an angle of about 25 degrees relative tolongitudinal axis 109. By way of further example, in certainembodiments, medical instrument 108 extends through side exit 105 alonga path at an angle of about 30 degrees relative to longitudinal axis109. By way of further example, in certain embodiments, medicalinstrument 108 extends through side exit 105 along a path at an angle ofabout 35 degrees relative to longitudinal axis 109. By way of furtherexample, in certain embodiments, medical instrument 108 extends throughside exit 105 along a path at an angle of about 40 degrees relative tolongitudinal axis 109. By way of further example, in certainembodiments, medical instrument 108 extends through side exit 105 alonga path at an angle of about 45 degrees relative to longitudinal axis109. By way of further example, in certain embodiments, medicalinstrument 108 extends through side exit 105 along a path at an angle ofabout 50 degrees relative to longitudinal axis 109. By way of furtherexample, in certain embodiments, medical instrument 108 extends throughside exit 105 along a path at an angle of about 55 degrees relative tolongitudinal axis 109. By way of further example, in certainembodiments, medical instrument 108 extends through side exit 105 alonga path at an angle of about 60 degrees relative to longitudinal axis109. By way of further example, in certain embodiments, medicalinstrument 108 extends through side exit 105 along a path at an angle ofabout 65 degrees relative to longitudinal axis 109. By way of furtherexample, in certain embodiments, medical instrument 108 extends throughside exit 105 along a path at an angle of about 70 degrees relative tolongitudinal axis 109.

Referring now to FIG. 39, another embodiment of the side exitingcatheter is shown wherein a side exiting tip component 101 is at distalend portion 134 of elongate flexible shaft 130. Side exiting tipcomponent 101 has a proximal and a distal end and a side exit 105. Incertain embodiments, localization element(s) 24 is positioned at or nearthe proximal end and/or near the distal end of side exiting tipcomponent 101. In this embodiment, medical instrument 108 extendsthrough channel 106 of side exiting tip component 101 and channel 107 ofelongate flexible shaft 130. Additionally, a medical instrument 108 isshown extended through side exit 105 along a path at an angle.

In yet another embodiment, side exiting tip component 101 may comprisemedical instrument 108 formed from a shape memory alloy whichtransitions between a first and second shape upon the application orrelease of stress. In certain embodiments, medical instrument 108 may bemade of a superelastic material such as Nickel-Titanium (Ni—Ti) alloy(commercially available as nitinol) that has a martensitic to austenitictransformation temperature below body temperature and below normal roomtemperature. In other embodiments, other suitable shape memory materialsfor medical instrument 108 can include elastic biocompatible metals suchas stainless steel, titanium, and tantalum or superelastic orpsuedoelastic copper alloys, such as Cu—Al—Ni, Cu—Al—Zi, and Cu—Zi. Whenformed, medical instrument 108 comprising shape memory alloy will have abend at a desired location with a bend angle (i.e., the first shape) andwhen housed within elongate flexible shaft 130, medical instrument 108becomes relatively straight (i.e., the second shape). When medicalinstrument 108 is advanced and extends through side exit 105, the stressis removed and medical instrument 108 will return back to its preformed(first) shape. Accordingly, medical instrument 108 comprising shapememory alloy may be able to interact with regions (or tissues) ofinterest at additional angles than can be achieved the by a medicalinstrument 108 comprising a non-shape memory material extending throughside exit 105. As illustrated in FIG. 40, the longitudinal axis of theportion of medical instrument 108 extending outside distal end portion134 is at an angle of approximately 90 degrees relative to thelongitudinal axis 109.

In yet another embodiment, medical instrument 108 comprising shapememory alloy may be used with a catheter having an exit at the distalend, wherein medical instrument 108 exits the catheter along alongitudinal axis. By using a medical instrument comprising a shapememory alloy with a catheter having an exit along a longitudinal axis, aphysician or other healthcare provider is able to target lesions, orother targets, that may not be directly in the airway or other pathways,but rather partially or even completely outside of the airway or otherpathways. Such targeting may not be possible with a catheter having anexit at the distal end and non-shape memory instruments. Similar to FIG.37, medical instrument 108 comprising shape memory alloy may comprise avariety of biopsy devices, including, but not limited to, for example, aforceps device 37A, an auger device 37D, a boring bit device 37B, abrush device 37C, and an aspiration needle device. In addition to biopsydevices, in yet other embodiments, a variety of shape memory instrumentsmay be used such as fiducial delivery devices, diagnostic imagingdevices (such as OCT), therapy devices (such as an ablation device, anRF emitter, a microwave emitter, a laser device, a device to deliver aradioactive seed, a cryogenic therapy delivery device, a drug deliverydevice, or a fluid delivery device, among others), or combinationsthereof, and the like.

As shown by FIGS. 41A, 41B, 41C and 41D, in various embodiments ofmedical instrument 108 used in steerable catheter 100, wherein medicalinstrument 108 is in the extended position it may be disposed at variousangles relative to longitudinal axis 109 of elongate flexible shaft 130at side exit 105. By way of example, in certain embodiments, whereinmedical instrument 108 is in the extended position it may be disposed atleast about 20 degrees relative to longitudinal axis 109 of elongateflexible shaft 130 at side exit 105. By way of further example, incertain embodiments, wherein medical instrument 108 is in the extendedposition it may be disposed at least about 30 degrees relative tolongitudinal axis 109 of elongate flexible shaft 130 at side exit 105.By way of further example, in certain embodiments, wherein medicalinstrument 108 is in the extended position it may be disposed at leastabout 40 degrees relative to longitudinal axis 109 of elongate flexibleshaft 130 at side exit 105. By way of further example, in certainembodiments, wherein medical instrument 108 is in the extended positionit may be disposed at least about 50 degrees relative to longitudinalaxis 109 of elongate flexible shaft 130 at side exit 105. By way offurther example, in certain embodiments, wherein medical instrument 108is in the extended position it may be disposed at least about 60 degreesrelative to longitudinal axis 109 of elongate flexible shaft 130 at sideexit 105. By way of further example, in certain embodiments whereinmedical instrument 108 is in the extended position it may be disposed atleast about 70 degrees relative to longitudinal axis 109 of elongateflexible shaft 130 at side exit 105. By way of further example, incertain embodiments, wherein medical instrument 108 is in the extendedposition it may be disposed at least about 80 degrees relative tolongitudinal axis 109 of elongate flexible shaft 130 at side exit 105.By way of further example, in certain embodiments wherein medicalinstrument 108 is in the extended position it may be disposed at leastabout 90 degrees relative to longitudinal axis 109 of elongate flexibleshaft 130 at side exit 105. By way of further example, in certainembodiments, wherein medical instrument 108 is in the extended positionit may be disposed at least about 100 degrees relative to longitudinalaxis 109 of elongate flexible shaft 130 at side exit 105. By way offurther example, in certain embodiments wherein medical instrument 108is in the extended position it may be disposed at least about 110degrees relative to longitudinal axis 109 of elongate flexible shaft 130at side exit 105. By way of further example, in certain embodiments,wherein medical instrument 108 is in the extended position it may bedisposed at least about 120 degrees relative to longitudinal axis 109 ofelongate flexible shaft 130 at side exit 105. By way of further example,in certain embodiments, wherein medical instrument 108 is in theextended position it may be disposed at least about 130 degrees relativeto longitudinal axis 109 of elongate flexible shaft 130 at side exit105. By way of further example, in certain embodiments, wherein medicalinstrument 108 is in the extended position it may be disposed at leastabout 140 degrees relative to longitudinal axis 109 of elongate flexibleshaft 130 at side exit 105. By way of further example, in certainembodiments, wherein medical instrument 108 is in the extended positionit may be disposed at least about 150 degrees relative to longitudinalaxis 109 of elongate flexible shaft 130 at side exit 105. By way offurther example, in certain embodiments, wherein medical instrument 108is in the extended position it may be disposed at least about 160degrees relative to longitudinal axis 109 of elongate flexible shaft 130at side exit 105. By way of further example, in certain embodiments,wherein medical instrument 108 is in the extended position it may bedisposed at least about 170 degrees relative to longitudinal axis 109 ofelongate flexible shaft 130 at side exit 105. By way of further example,in certain embodiments, wherein medical instrument 108 is in theextended position it may be disposed at about 180 degrees relative tolongitudinal axis 109 of elongate flexible shaft 130 at side exit 105.

As shown in FIG. 17G, in an embodiment of the present invention,elongate flexible shaft 130 of side exiting catheter 100 can besteerable wherein elongate flexible shaft 130 comprises a steeringmechanism comprising a steering actuator 218 at proximal end portion 132and a pull wire connected to steering actuator 218 wherein distal endportion 134 may be moved relative to proximal end portion 132 bymanipulating steering actuator 218 to apply a tension to the pull wire.Referring now to FIG. 41E, medical instrument 108 is shown in theextended position through side exit 105 of a steerable cathetercomprising elongate flexible shaft 230. Medical instrument 108 extendsthrough side exit 105 along a path at an angle Θ of at least 10 degreesrelative to longitudinal axis 207. As described in greater detailelsewhere herein, in one embodiment, manipulation of steering actuator218 (see FIGS. 18A and 18B) introduces an arc of β degrees into elongateflexible shaft 230. Distal end portion 234 may be moved to a distancethat separates two regions of elongate flexible shaft 230 located atopposing ends of a chord X connecting to two points separated by βdegrees on the arc.

Typically, the outer diameter of elongate flexible shaft 130 of sideexiting catheter 100 is less than 5 mm. By way of example, in certainembodiments, the outer diameter elongate flexible shaft 130 of sideexiting catheter 100 is less than 1 mm. By way of further example, incertain embodiments, the outer diameter of elongate flexible shaft 130of side exiting catheter 100 is less than 2 mm. By way of furtherexample, in certain embodiments, the outer diameter of elongate flexibleshaft 130 of side exiting catheter 100 is less than 3 mm. By way offurther example, in certain embodiments, the outer diameter of elongateflexible shaft 130 of side exiting catheter 100 is less than 4 mm. Byway of further example, in certain embodiments, the outer diameter ofelongate flexible shaft 130 of side exiting catheter 100 is less than 5mm.

However, in other embodiments, in addition to, or in place of,localization element 24, side exiting catheter 100 may be navigatedwherein elongate flexible shaft 130 or medical instrument 108 mayfurther comprise radiopaque markers visible via fluoroscopic imaging, orechogenic materials or patterns that increase visibility of the tipcomponent under an ultrasonic beam. In yet other embodiments, sideexiting catheter 100 may be navigated via other types of sensors, suchas conductive localization elements, fiber optic localization elements,or any other type of localization element.

In one embodiment, side exiting tip component 101, illustrated by FIGS.42A, 42B, and 42C, may be visible via fluoroscopic imaging. An angled ordirectionally arranged radiopaque marker pattern 115 at or near theproximal end and/or the distal end of side exiting tip component 101.The radiopaque marker pattern 115 may be made of stainless steel,tantalum, platinum, gold, barium, bismuth, tungsten, iridium, orrhenium, alloys thereof, or of other radiopaque materials known in theart. Radiopaque marker pattern 115 allows for tracking of the locationand orientation of side exit 105. Based upon the orientation ofradiopaque marker pattern 115 with respect to the incident fluoroscopicbeam, a distinct image is visible on a fluoroscope. As side exitingcatheter 100 is navigated and rotated into position at the region (ortissue of interest), the orientation of radiopaque marker pattern 115with respect to the incident fluoroscopic beam will be altered resultingin a corresponding change to the fluoroscopic image, thereby allowingthe physician or other healthcare professional to know the location andorientation of side exit 105. Accordingly, the physician or otherhealthcare professional can then extend medical instrument 108 at theregion (or tissue) of interest. Additionally, radiopaque marker pattern115 may assist the physician or other healthcare professional inavoiding the passage of the elongate flexible shaft 130 and the medicalinstrument 108 into healthy tissue, a blood vessel, or other undesiredpatient area. In other embodiments, radiopaque marker pattern 115 may beon distal end portion 134 of elongate flexible shaft 130 of side exitingcatheter 100. In one embodiment radiopaque marker pattern 115 mayreplace a six (6) degree of freedom (6DOF) electromagnetic sensor. Inother embodiments, radiopaque marker pattern 115, may supplement a six(6) degree of freedom (6DOF) electromagnetic sensor.

As illustrated in FIGS. 43A, 43B, and 43C, in another embodimentradiopaque marker pattern 115 comprises generally circular radiopaquemarkers 116 placed around side exiting tip component 101, with 3radiopaque markers on one side of side exiting tip component 101 and 1radiopaque marker on the opposite side of side exiting tip component101. By way of example, in certain embodiments, 4 radiopaque markers areplaced on one side of side exiting tip component 101 and 2 radiopaquemarkers are placed on the opposite side of side exiting tip component101. By way of further example, in certain embodiments, 5 radiopaquemarkers are placed on one side of side exiting tip component 101 and 3radiopaque markers are placed on the opposite side of side exiting tipcomponent 101. In other embodiments, radiopaque marker pattern 115 maybe on distal end portion 134 of elongate flexible shaft 130 of sideexiting catheter 100.

In yet another embodiment, at least one ring of radiopaque material 115may surround side exit 105. By way of example, in one embodimentillustrated in FIGS. 44A, 44B, and 44C, side exit 105 of side exitingtip component 101 is surrounded by a first ring of radiopaque material115 that is concentric to and in immediate connection with side exit 105of the side exiting tip component 101. This first ring of radiopaquematerial 115 is further surrounded by additional separate rings that areconcentric to side exit 105 and the first ring of radiopaque material115. By way of example, in certain embodiments, side exit 105 issurrounded by 1 ring of radiopaque material 115. By way of furtherexample, in certain embodiments, side exit 105 is surrounded by 2 ringsof radiopaque material 115. By way of further example, in certainembodiments, side exit 105 is surrounded by 3 rings of radiopaquematerial 115. By way of further example, in certain embodiments, sideexit 105 is surrounded by 4 rings of radiopaque material 115. By way offurther example, in certain embodiments, side exit 105 is surrounded by5 rings of radiopaque material 115. By way of further example, incertain embodiments, side exit 105 is surrounded by 6 rings ofradiopaque material 115. In other embodiments, radiopaque marker pattern115 may be on distal end portion 134 of elongate flexible shaft 130.

In certain embodiments, all or a portion of side exiting tip component101 and/or distal end portion 134 of elongate flexible shaft 130 of sideexiting catheter 100 may be echogenic such that it may be viewed viaultrasonic imaging. Several approaches of enhancing the ultrasonicsignature of medical devices through modification of the device surfacereflectivity are known in the prior art and can be applied to certainembodiments of the present invention. In one embodiment, an echogenicpattern can be positioned around the side exiting tip component 101and/or around distal end portion 134 of elongate flexible shaft 130,such that the echogenic pattern covers the exterior circumference ofside exiting tip component 101 and/or distal end portion 134 of elongateflexible shaft 130. Typically an echogenic pattern is on the exteriorsurface of side exiting tip component 101 defined as a length from thedistal end of side exiting tip component 101 toward the proximal end ofside exiting tip component 101. By way of example, in one embodiment,the echogenic pattern has a length of about 1 cm from the distal end ofside exiting tip component 101. By way of further example, in anotherembodiment, the echogenic pattern has a length of about 2 cm from distalend of the side exiting tip component 101.

In one embodiment, as illustrated by FIG. 45 and FIG. 45A, the echogenicpattern comprises a plurality of grooves 113 are cut into the exteriorsurface of side exiting tip component 101 such that the grooves increasethe reflective coefficient of side exiting tip component 101. Theplurality of grooves 113 cause constructive interference of theultrasonic beam to affect an amplification of the reflecting beam alongthe line of the incident beam. In other embodiments, plurality ofgrooves 113 may be in distal end portion 134 of elongate flexible shaft130 of side exiting catheter 100.

By way of further example, another embodiment of side exiting tipcomponent 101 with an echogenic pattern is shown in FIG. 46 and FIG.46A. In this embodiment, the echogenic pattern comprises a plurality ofpartially spherical indentations 114 on the exterior surface of sideexiting tip component 101, such that the radius of the partiallyspherical indentations is less than the wavelength of the incidentultrasonic beam. The plurality of partially spherical indentations 114cause constructive interference of the ultrasonic beam to affect anamplification of the reflecting beam along the line of the incidentbeam; such amplification occurs at any incident ultrasonic beam angle.In other embodiments, plurality of partially spherical indentations 114may be in distal end portion 134 of elongate flexible shaft 130 of sideexiting catheter 100.

In certain embodiments, as shown in FIG. 47, all or a portion of medicalinstrument 108 may be echogenic such that it may be viewed viaultrasonic imaging. Several approaches of enhancing the ultrasonicsignature of medical devices through modification of the device surfacereflectivity are known in the prior art and can be applied to certainembodiments of the present invention.

In certain embodiments, as discussed herein, localization element 24 ispositioned at or near distal end portion 134 of elongate flexible shaft130. In one embodiment, as shown in FIG. 48, localization element 24comprises a six (6) degree of freedom (6DOF) electromagnetic sensor inthe distal end portion 134 of the elongate flexible shaft 130. Alocalization element lead 103 extends from localization element 24 toproximal end portion 132 (not shown) of elongate flexible shaft 130. Byway of further example, in another embodiment, an angled or directionalpattern may be on the outside of elongate flexible shaft 130 that isvisible via fluoroscopic imaging. Accordingly, embodiments of thepresent invention are not limited to one type and position oflocalization elements 24. In certain embodiments, a combination ofdifferent localization element 24 types or a series of the samelocalization element 24 types may positioned at or near distal endportion 134 of elongate flexible shaft 130 of side exiting catheter 100.

Another embodiment, with a radiopaque marker pattern positioned at ornear the distal portion 134 of the elongate flexible shaft 130 isillustrated by FIGS. 49A, 49B, and 49C. In this embodiment a radiopaquemarker pattern 115 comprising generally circular radiopaque markers 116may be placed around distal end portion 134 of elongate flexible shaft130, with 3 markers on one side of the distal end portion 134 ofelongate flexible shaft 130 and 1 marker on the opposite side of distalend portion 134 of elongate flexible shaft 130. By way of example, incertain embodiments, 4 markers are placed on one side of distal endportion 134 of elongate flexible shaft 130 and 2 markers are placed onthe opposite side of distal end portion 134 of elongate flexible shaft130. By way of example, in certain embodiments, 5 markers are placed onone side of distal end portion 134 of elongate flexible shaft 130 and 3markers are placed on the opposite side of distal end portion 134 ofelongate flexible shaft 130 of side exiting catheter 100.

A method or procedure of guiding the side exiting catheter 100 ofcertain embodiments to a desired target tissue in the respiratory systemof a patient may comprise displaying an image of the region of thepatient, inserting a flexible lumen into the patient, and inserting intothe flexible lumen side exiting catheter 100. Side exiting catheter 100typically comprises an electromagnetic localization element at distalend portion 134. Side exiting catheter may then be navigated to theregion of interest and the location and orientation of theelectromagnetic localization element is detected. Then medicalinstrument 108 may be displayed, in real-time, on the image bysuperimposing a virtual representation of side exiting catheter 100 andmedical instrument 108 on the image based upon the location andorientation of localization element 24. Then a medical procedure may beperformed at the region (or tissue) of interest. In certain embodiments,side exiting catheter 100 further comprises an elongate flexible shaft130 having a proximal end portion 132, an opposite distal end portion134, a longitudinal axis 109, and an outer wall 136 comprising abiocompatible material extending from proximal end portion 132 to distalend portion 134. Side exiting catheter 100 may also comprise a handle110 attached to proximal end portion 132, and a medical instrument 108housed within distal end portion 134 of elongate flexible shaft 130 thatis extendable along a path from a position within outer wall 136 throughside exit 105 to a position outside outer wall 136 at an angle of atleast 30 degrees relative to the longitudinal axis 109.

As illustrated in FIGS. 50A and 50B, another embodiment of the presentinvention includes an offset device to control the extension of a biopsydevice or medical instrument. The offset device may comprise, forexample, features (e.g., offsets) that snap-on or may be otherwiseaffixed to handle 216 that are capable of holding a biopsy device 220 ina known or preset (i.e., predetermined) location to maintain the biopsydevice 220 extension and free the hands of the physician or otherhealthcare professional. In one embodiment, for example, the offsetdevice includes one or more offset portions that can be adjusted bycombining and/or removing multiple offset segments. In anotherembodiment, for example, the offset device includes one or more offsetportions that can be adjusted by the removal of one or more removableoffset segments separated by perforations (i.e., in a disposablefashion). In yet another embodiment, the offset device includes anoffset that is capable of adjustment using a screw mechanism (i.e., thelength of the offset can be adjusted by screwing the offset in and out).In various embodiments, each offset can be represented on the navigationscreen showing offset distance from the distal end portion of theelongate flexible shaft. This representation is accomplished via sensorson the offset devices that would register the selected offset position.In additional embodiments, sensors (e.g., electromagnetic sensors,proximity sensors, limit switch sensors) at the handle may register thepresence or contact with an offset device such that the extension depthof the biopsy device can be measured and represented on the navigationscreen. The surgical catheter, in various embodiments, may include oneor more offsets, two or more offsets, three or more offsets, four ormore offsets, or five or more offsets. In other embodiments, more thanfive offsets may be included (e.g., 6-12 offsets, 6-18 offsets, 6-24offsets, or more). An embodiment of offset device wherein the biopsydevice is a brush is shown in FIGS. 50A and 50B. Another embodiment ofthe offset device wherein the medical instrument is a side exiting tipcomponent is shown in FIGS. 51A and 51B.

As illustrated in FIGS. 50A and 50B, handle 216 attached to elongateflexible shaft 230 of steerable catheter 200 may include one or moreoffset devices 1501. The offset device(s) 1501 may be capable of holdingthe biopsy device 220, depicted as a brush, in place and provide asubstantially fixed distance or length of extension out of elongateflexible shaft 230 to take a biopsy. As shown, multiple offsets 1501 canbe provided in stages that allow extension of biopsy device 220 (e.g.,at 1 cm, 2 cm, 3 cm, and so on) whereby the physician or otherhealthcare professional can adjust the offsets by removal orrepositioning (removed offsets 1501R, for example, are depicted indashed lines within the port in FIG. 50B). Thus, FIG. 50A shows biopsydevice 220 prior to extension, whereas FIG. 50B shows biopsy deviceextended (e.g., 2 cm of extension) by the removal or repositioning oftwo offsets 1501R.

As shown in FIGS. 51A and 51B, another embodiment of the presentinvention is directed to an offset device for use with a medicalinstrument 108 that exits the side of elongate flexible shaft 130 ofside exiting catheter 100. Handle 110 attached to elongate flexibleshaft 130 may include one or more offset devices 1501. The offsetdevice(s) 1501 may be capable of holding the medical instrument 108,depicted as an aspiration needle, in place and provide a substantiallyfixed distance or length of extension out of elongate flexible shaft 130to take a biopsy. As shown, multiple offsets 1501 can be provided instages that allow extension of medical instrument 108 (e.g., at 1 cm, 2cm, 3 cm, and so on) out the side exit 105 of the elongate flexibleshaft 130 whereby the physician or other healthcare professional canadjust the offsets by removal or repositioning (removed offsets 1501R,for example, are depicted in dashed lines within the port in FIG. 51B).Thus, FIG. 51A shows medical instrument 108 prior to extension, whereasFIG. 51B shows medical instrument 108 extended (e.g., 2 cm of extension)out side exit 105 of the elongate flexible shaft 130 by the removal orrepositioning of two offsets 1501R.

Typically, biopsy device 220 may be extended a distance (extendeddistance) correlated to movement of handle 216 wherein extended distancemay be from about 0.5 cm to about 4.0 cm. By way of example, in certainembodiments, the extended distance is at least about 0.5 cm. By way offurther example, in certain embodiments, the extended distance is atleast about 1.0 cm. By way of further example, in certain embodiments,the extended distance is at least about 1.5 cm. By way of furtherexample, in certain embodiments, the extended distance is at least about2.0 cm. By way of further example, in certain embodiments, the extendeddistance is at least about 2.5 cm. By way of further example, in certainembodiments, the extended distance is at least about 3.0 cm. By way offurther example, in certain embodiments, the extended distance is atleast about 3.5 cm. By way of further example, in certain embodiments,the extended distance is about 4.0 cm.

Typically, medical instrument 108 may be advanced a distance (advanceddistance) correlated to movement of handle 110 wherein advanced distancemay be from about 0.5 cm to about 4.0 cm. By way of example, in certainembodiments, the advanced distance is at least about 0.5 cm. By way offurther example, in certain embodiments, the advanced distance is atleast about 1.0 cm. By way of further example, in certain embodiments,the advanced distance is at least about 1.5 cm. By way of furtherexample, in certain embodiments, the advanced distance is at least about2.0 cm. By way of further example, in certain embodiments, the advanceddistance is at least about 2.5 cm. By way of further example, in certainembodiments, the advanced distance is at least about 3.0 cm. By way offurther example, in certain embodiments, the advanced distance is atleast about 3.5 cm. By way of further example, in certain embodiments,the advanced distance is about 4.0 cm.

In another embodiment, as shown in FIGS. 52A, 52B, 52C, and 52D, alocalization element 24 (e.g., electromagnetic sensor) can be attachedto an existing, non-navigated surgical instrument or device 70 (e.g.,steerable catheter, non-steerable catheter, bronchoscope, forcepsdevice, auger device, boring bit device, aspiration needle device, brushdevice, side exiting tip component, etc.) for use in the systems andmethods described herein. In one embodiment, as illustrated in FIG. 52A,a plastic or polymer sheath or condom 72 is placed over a localizationelement lead wire 103 and localization element 24. Then, as illustratedin FIG. 52B, the sheath or condom 72 may then be shrink-fitted tosurgical instrument or device 70 via the application of a temperaturedifferential. Typically the application of a heat treatment causessheath or condom 72 to shrink to surgical instrument or device 70 andlocalization element lead wire/localization element combination, thusconverting existing, non-navigated surgical instrument or device 70 to anavigated surgical instrument or device 70. While heat treating is acommon way to apply a temperature differential, it is understood thatthe existing, non-navigated surgical instrument or device 70 can becooled before the plastic or polymer sheath or condom 72 is placed overexisting, non-navigated surgical instrument or device 70. In anotherembodiment, as shown in FIG. 52C more than one localization element 24may be attached to an existing, non-navigated surgical instrument ordevice 70 using a stretchable plastic or polymer sheath or condom 72. Asillustrated in FIG. 52D, the stretchable plastic or polymer sheath orcondom 72 may extend past the end of surgical instrument or device 70such that a lip 74 is created when the sheath or condom 72 is affixed tonon-navigated surgical instrument or device 70. This lip 74 may assistin registration of surgical instrument or device 70. In otherembodiments, localization element 24 may be wireless, such that alocalization element lead wire 103 is not required.

Other embodiments include, for example, a plastic or polymer sheath orcondom that is custom sized to fit over an existing, non-navigatedsurgical instrument or device 70 and may be placed over a localizationelement lead wire 103 and localization element 24 to add a localizationelement 24 (e.g., an electromagnetic sensor), thus converting existing,non-navigated surgical instrument or device 70 to a navigated surgicalinstrument or device 70. In this embodiment, the plastic or polymersheath or condom may be held in place on the existing, non-navigatedsurgical instrument or device 70 by a friction fit. In yet otherembodiments, an elastic or stretchable plastic or polymer sheath orcondom may be expanded and placed over a localization element lead wire103 and localization element 24 to add a localization element 24 (e.g.,an electromagnetic sensor), thus converting the existing, non-navigatedsurgical instrument or device 70 to a navigated surgical instrument ordevice 70. In this embodiment, the elastic or stretchable plastic orpolymer sheath or condom may also be held in place on the existing,non-navigated surgical instrument or device 70 by a friction fit.

In yet other embodiments, a localization element 24 may be affixed to anexisting, non-navigated surgical instrument or device 70 with tape. Incertain embodiments, localization element 24 may be wireless. In otherembodiments, localization element 24 may be affixed to an existing,non-navigated surgical instrument or device 70 via an adhesive.

In addition to or in place of localization element 24, steerablecatheter 200 and/or side exiting catheter 100 may be equipped with oneor more sensing devices at or near the distal end portion of theelongate flexible shaft and/or at the biopsy device 220 or medicalinstrument 108 of the respective catheters described herein. Additionalsensing devices may include electrodes for sensing depolarizationsignals occurring in excitable tissue such as the heart, nerve or brain.In one embodiment, for use in cardiac applications, the sensing devicemay include at least one electrode for sensing internal cardiacelectrogram (EGM) signals. In other embodiments, the sensing device maybe an absolute pressure sensor to monitor blood pressure. In still otherembodiments, surgical instrument 12 may be equipped with other sensingdevices including physiological detection devices, localizationelements, temperature sensors, motion sensors, optical coherencetomography (OCT) sensors, endobronchial ultrasound (EBUS) sensors, orDoppler or ultrasound sensors that can detect the presence or absence ofblood vessels.

The accompanying Figures and this description depict and describecertain embodiments of a navigation system (and related methods anddevices) in accordance with the present invention, and features andcomponents thereof. It should also be noted that any references hereinto front and back, right and left, top and bottom and upper and lowerare intended for convenience of description, not to limit the presentinvention or its components to any one positional or spatialorientation.

It is noted that the terms “comprise” (and any form of comprise, such as“comprises” and “comprising”), “have” (and any form of have, such as“has” and “having”), “contain” (and any form of contain, such as“contains” and “containing”), and “include” (and any form of include,such as “includes” and “including”) are open-ended linking verbs. Thus,a method, an apparatus, or a system that “comprises,” “has,” “contains,”or “includes” one or more items possesses at least those one or moreitems, but is not limited to possessing only those one or more items

Individual elements or steps of the present methods, apparatuses, andsystems are to be treated in the same manner. Thus, a step that callsfor modifying a segmented image dataset for a region of a respiratorysystem to match the corresponding anatomy of a patient's respiratorysystem, that includes the steps of: (i) forming a respiratory-gatedpoint cloud of data that demarcates anatomical features in a region of apatient's respiratory system at one or more discrete phases within arespiration cycle of a patient, (ii) density filtering therespiratory-gated point cloud, (iii) classifying the density filteredrespiratory-gated point cloud according to anatomical points ofreference in a segmented image dataset for the region of the patient'srespiratory system, and (iv) modifying the segmented image dataset tocorrespond to the classified anatomical points of reference in thedensity filtered respiratory-gated point cloud, but also covers thesteps of (i) comparing the registered respiratory-gated point cloud to asegmented image dataset to determine the weighting of points comprisedby the classified respiratory-gated point cloud, (ii) distinguishingregions of greater weighting from regions of lesser weighting, and (iii)modifying the segmented image dataset to correspond to the classifiedrespiratory-gated point cloud.

The terms “a” and “an” are defined as one or more than one. The term“another” is defined as at least a second or more. The term “coupled”encompasses both direct and indirect connections, and is not limited tomechanical connections.

Those of skill in the art will appreciate that in the detaileddescription above, certain well known components and assembly techniqueshave been omitted so that the present methods, apparatuses, and systemsare not obscured in unnecessary detail.

While various embodiments of the invention have been described above, itshould be understood that they have been presented by way of exampleonly, and not limitation. Thus, the breadth and scope of the inventionshould not be limited by any of the above-described embodiments, butshould be defined only in accordance with the following claims and theirequivalents.

The previous description of the embodiments is provided to enable anyperson skilled in the art to make or use the invention. While theinvention has been particularly shown and described with reference toembodiments thereof, it will be understood by those skilled in art thatvarious changes in form and details may be made therein withoutdeparting from the spirit and scope of the invention. For example, theelongate flexible shafts, biopsy device, medical instruments, andlocalization elements can be constructed from any suitable material, andcan be a variety of different shapes and sizes, not necessarilyspecifically illustrated, while still remaining within the scope of theinvention.

What is claimed is:
 1. A method of modifying a segmented image datasetfor a region of a respiratory system to match the corresponding anatomyof a patient's respiratory system, the method comprising: (i) forming arespiratory-gated point cloud of data by moving a catheter having afiber optic localization element through a plurality of locations in thepatient's respiratory system, wherein the respiratory-gated point cloudof data demarcates anatomical features in a region of a patient'srespiratory system at one or more discrete phases within a respirationcycle of the patient, (ii) density filtering the respiratory-gated pointcloud, (iii) classifying the density filtered respiratory-gated pointcloud according to anatomical points of reference in a segmented imagedataset for the region of the patient's respiratory system, (iv)continuously modifying the segmented image dataset to correspond to theclassified anatomical points of reference in the density filteredrespiratory-gated point cloud to produce simulated real-time,intra-procedural images illustrating the orientation and shape of theanatomical features in the region of a patient's respiratory system, and(vii) detecting center locations of selected imaged anatomical featuresby applying a centroid finding algorithm to selected points of thedensity filtered respiratory-gated point cloud.
 2. The method of claim 1wherein the discrete phases are expiration, inspiration and discretephases there-between.
 3. The method of claim 1 wherein the segmentedimage dataset is from a first discrete phase of the patient'srespiration cycle and the respiratory-gated point cloud is from a secondand different discrete phase of the patient's respiration cycle.
 4. Themethod of claim 1 wherein the segmented image dataset is a skeletonizedsegmented image dataset.
 5. A method of simulating the movement of apatient's respiratory system in the patient's respiration cycle duringrespiration comprising: (i) forming a respiratory-gated point cloud ofdata by moving a catheter having a fiber optic localization elementthrough a plurality of locations in the patient's respiratory system,wherein the respiratory-gated point cloud of data demarcates anatomicalfeatures in a region of a patient's respiratory system at one or morediscrete phases within a respiration cycle of a patient, (ii) densityfiltering the respiratory-gated point cloud, (iii) classifying thedensity filtered respiratory-gated point cloud according to anatomicalpoints of reference in a segmented image dataset for the region of thepatient's respiratory system, (iv) creating a cine loop comprising aplurality of modified segmented image datasets through multiplemodifications of the segmented image dataset to correspond to aplurality of classified anatomical points of reference in therespiratory-gated point cloud over the respiration cycle, (v) displayingthe cine loop comprising the plurality of modified segmented imagedatasets over the patient's respiration cycle, and (vi) detecting centerlocations of selected imaged anatomical features by applying a centroidfinding algorithm to selected points of the density filteredrespiratory-gated point cloud.
 6. The method of simulation of claim 5wherein displaying the cine loop comprising the plurality of modifiedsegmented image datasets over the patient's respiration cycle issynchronized with the patient's respiration cycle.
 7. A method ofpreparing a segmented image dataset to match the anatomy of a patient'srespiratory system, the method comprising: (i) forming arespiratory-gated point cloud of data by moving a catheter having afiber optic localization element through a plurality of locations in thepatient's respiratory system, wherein the respiratory-gated point cloudof data demarcates anatomical features in a region of a patient'srespiratory system at one or more discrete phases within a respirationcycle of the patient, (ii) density filtering the respiratory-gated pointcloud, (iii) classifying the density filtered respiratory-gated pointcloud according to anatomical points of reference in a segmented imagedataset for the region of the patient's respiratory system, (iv)registering the classified respiratory-gated point cloud to thesegmented image dataset, (v) comparing the classified respiratory-gatedpoint cloud to a segmented image data set to determine the weighting ofpoints comprised by the respiratory-gated point cloud, (vi)distinguishing regions of greater weighting from regions of lesserweighting and optionally increasing the data set comprised by therespiratory-gated point cloud for regions of lesser weighting, (vii)continuously modifying the registered segmented image dataset tocorrespond to the classified anatomical points of reference in therespiratory-gated point cloud to produce simulated real-time,intra-procedural images illustrating the orientation and shape of theanatomical features in the region of a patient's respiratory system, and(viii) detecting center locations of selected imaged anatomical featuresby applying a centroid finding algorithm to selected points of therespiratory-gated point cloud.
 8. The method of claim 7 whereinregistering the respiratory-gated point cloud to the segmented imagedata set comprises: registering the respiratory-gated point cloudrepresenting at least one branch of the patient's respiratory system tocorresponding anatomical points of reference in the registered segmentedimage data set representing the branch(es) of the patient's respiratorysystem.
 9. The method of claim 7 wherein the respiratory-gated pointcloud is registered to a plurality of branches of the patient'srespiratory system, wherein the plurality of branches comprise thetrachea, the right main bronchus, and the left main bronchus.
 10. Themethod of claim 7 wherein the discrete phases are expiration,inspiration and discrete phases there-between.
 11. The method of claim 7wherein the segmented image dataset is from a first discrete phase ofthe patient's respiration cycle and the respiratory-gated point cloud isfrom a second and different discrete phase of the patient's respirationcycle.
 12. A non-transitory processor-readable medium storing coderepresenting instructions to cause a processor to perform a process, thecode comprising code to carry out one or more elements of the method ofclaim
 1. 13. A non-transitory processor-readable medium storing coderepresenting instructions to cause a processor to perform a process, thecode comprising code to carry out one or more elements of the method ofclaim
 5. 14. A non-transitory processor-readable medium storing coderepresenting instructions to cause a processor to perform a process, thecode comprising code to carry out one or more elements of the method ofclaim 7.